-- Logs begin at Fri 2022-08-05 18:47:39 UTC, end at Fri 2022-08-05 20:26:30 UTC. -- Aug 05 19:48:23.603463 dv-cinder-852210-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 05 19:48:23.610980 dv-cinder-852210-2-iscsi devstack@keystone.service[80468]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 05 19:48:23.616673 dv-cinder-852210-2-iscsi devstack@keystone.service[80468]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Aug 5 19:48:23 2022] *** Aug 05 19:48:23.616673 dv-cinder-852210-2-iscsi devstack@keystone.service[80468]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 05 19:48:23.616673 dv-cinder-852210-2-iscsi devstack@keystone.service[80468]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 05 19:48:23.616673 dv-cinder-852210-2-iscsi devstack@keystone.service[80468]: nodename: dv-cinder-852210-2-iscsi Aug 05 19:48:23.616673 dv-cinder-852210-2-iscsi devstack@keystone.service[80468]: machine: x86_64 Aug 05 19:48:23.616673 dv-cinder-852210-2-iscsi devstack@keystone.service[80468]: clock source: unix Aug 05 19:48:23.616673 dv-cinder-852210-2-iscsi devstack@keystone.service[80468]: pcre jit disabled Aug 05 19:48:23.616673 dv-cinder-852210-2-iscsi devstack@keystone.service[80468]: detected number of CPU cores: 4 Aug 05 19:48:23.616673 dv-cinder-852210-2-iscsi devstack@keystone.service[80468]: current working directory: / Aug 05 19:48:23.616673 dv-cinder-852210-2-iscsi devstack@keystone.service[80468]: detected binary path: /usr/bin/uwsgi-core Aug 05 19:48:23.616673 dv-cinder-852210-2-iscsi devstack@keystone.service[80468]: your processes number limit is 47680 Aug 05 19:48:23.616673 dv-cinder-852210-2-iscsi devstack@keystone.service[80468]: your memory page size is 4096 bytes Aug 05 19:48:23.616673 dv-cinder-852210-2-iscsi devstack@keystone.service[80468]: detected max file descriptor number: 2048 Aug 05 19:48:23.616673 dv-cinder-852210-2-iscsi devstack@keystone.service[80468]: lock engine: pthread robust mutexes Aug 05 19:48:23.616673 dv-cinder-852210-2-iscsi devstack@keystone.service[80468]: thunder lock: enabled Aug 05 19:48:23.616673 dv-cinder-852210-2-iscsi devstack@keystone.service[80468]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 05 19:48:23.616673 dv-cinder-852210-2-iscsi devstack@keystone.service[80468]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 05 19:48:23.655582 dv-cinder-852210-2-iscsi devstack@keystone.service[80468]: Python main interpreter initialized at 0x55ca2d41c9c0 Aug 05 19:48:23.655582 dv-cinder-852210-2-iscsi devstack@keystone.service[80468]: python threads support enabled Aug 05 19:48:23.655582 dv-cinder-852210-2-iscsi devstack@keystone.service[80468]: your server socket listen backlog is limited to 100 connections Aug 05 19:48:23.655582 dv-cinder-852210-2-iscsi devstack@keystone.service[80468]: your mercy for graceful operations on workers is 90 seconds Aug 05 19:48:23.655948 dv-cinder-852210-2-iscsi devstack@keystone.service[80468]: mapped 403077 bytes (393 KB) for 2 cores Aug 05 19:48:23.656028 dv-cinder-852210-2-iscsi devstack@keystone.service[80468]: *** Operational MODE: preforking *** Aug 05 19:48:23.656278 dv-cinder-852210-2-iscsi devstack@keystone.service[80468]: *** uWSGI is running in multiple interpreter mode *** Aug 05 19:48:23.656278 dv-cinder-852210-2-iscsi devstack@keystone.service[80468]: spawned uWSGI master process (pid: 80468) Aug 05 19:48:23.656487 dv-cinder-852210-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 05 19:48:23.658671 dv-cinder-852210-2-iscsi devstack@keystone.service[80468]: spawned uWSGI worker 1 (pid: 80477, cores: 1) Aug 05 19:48:23.658671 dv-cinder-852210-2-iscsi devstack@keystone.service[80468]: spawned uWSGI worker 2 (pid: 80478, cores: 1) Aug 05 19:48:23.658671 dv-cinder-852210-2-iscsi devstack@keystone.service[80468]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 05 19:48:29.185335 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 05 19:48:29.185335 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: from cryptography.utils import int_from_bytes Aug 05 19:48:29.186039 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 05 19:48:29.186039 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: from cryptography.utils import int_from_bytes Aug 05 19:48:29.187125 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 05 19:48:29.187125 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: from cryptography.utils import int_from_bytes Aug 05 19:48:29.187576 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 05 19:48:29.187576 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: from cryptography.utils import int_from_bytes Aug 05 19:48:29.235548 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 05 19:48:29.235726 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 05 19:48:29.235912 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] command line args: [] {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 05 19:48:29.236180 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 05 19:48:29.236358 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ================================================================================ {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 05 19:48:29.236637 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] admin_token = **** {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.236716 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 05 19:48:29.236916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 05 19:48:29.237032 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] config_dir = [] {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.237106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] command line args: [] {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 05 19:48:29.237165 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.237343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] config_source = [] {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.237455 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 05 19:48:29.237542 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] debug = True {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.237633 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ================================================================================ {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 05 19:48:29.237693 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.237936 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] admin_token = **** {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.238028 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] config_dir = [] {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.238100 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.238220 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.238284 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.238401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] config_source = [] {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.238574 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.238689 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] debug = True {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.238840 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.238946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.239074 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] list_limit = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.239176 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.239275 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] log_config_append = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.239367 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.239458 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.239556 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.239657 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] log_dir = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.239657 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] log_file = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.239859 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.239961 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.240052 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] list_limit = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.240052 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] log_config_append = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.240239 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.240338 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.240439 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.240532 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] log_dir = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.240532 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] log_file = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.240739 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.240739 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.240948 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.241060 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.241156 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.241246 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.241338 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.241438 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.241540 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.241636 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.241728 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.241826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.241931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.242025 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.242123 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.242223 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.242322 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.242516 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.242633 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.242734 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.242833 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.242951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.243075 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.243177 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.243422 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.243534 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.39/identity {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.243668 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] publish_errors = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.243819 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.243907 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.244050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.244133 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.244269 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.244457 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.244612 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.39/identity {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.244698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.244805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] publish_errors = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.244916 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] standard_threads = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.245042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.245138 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.245277 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.245359 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.245458 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.245584 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.245690 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] use_journal = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.245765 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.245881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] use_json = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.245953 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.246065 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] use_stderr = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.246152 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] standard_threads = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.246223 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] use_syslog = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.246367 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.246489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.246610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.246685 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.246788 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.246864 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.247105 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] use_journal = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.247180 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.247294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] use_json = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.247294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] use_stderr = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.247476 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.247549 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] use_syslog = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.247667 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.247740 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:29.247814 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.247932 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.248013 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.248152 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.248233 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.248416 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.248499 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.248684 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.248810 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.248934 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.249043 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.249162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.249243 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.249407 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.249494 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.249643 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.249723 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.249886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.249966 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.250131 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.250220 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.250367 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.250453 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.250601 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.250681 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.250849 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.251002 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.251079 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.251154 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.251313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.251393 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.251559 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.251648 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.251798 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.251873 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.252054 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.252135 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.252297 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.252381 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.253108 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.253333 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.253409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.253587 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.253716 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.253842 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.253960 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.254082 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.254206 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.254330 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.254460 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.254582 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.254716 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.254839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.254959 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.255086 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.255215 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.255339 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.255466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.255589 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.255709 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.255849 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.255972 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.255972 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.256196 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.256323 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.256445 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.256566 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.256734 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.256852 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.256974 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.257095 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.257222 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.257344 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.257540 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.257665 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.257792 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.257914 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.258036 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.258167 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.258298 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.258427 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.258555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.258677 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.258799 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.258921 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.259081 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.259212 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.259336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.259457 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.259579 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.259700 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.259939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.260077 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.260241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.260373 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.260498 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.260676 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.260819 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.260945 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.261069 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.261192 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.261366 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.261551 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.261696 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.261823 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.261958 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.262053 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.262173 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.262326 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.262449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.262632 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.262699 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.262764 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.262832 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.262897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.262963 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.263028 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.263087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.263150 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.263274 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.263354 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.263425 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.263545 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.263625 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.263701 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.263822 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.263895 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.263974 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.264052 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.264178 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.264252 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.264326 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.264440 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.264513 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.264652 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.264770 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.264844 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.264956 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.265030 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.265142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.265223 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.265294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.265406 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.265479 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.265591 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.265664 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.265784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.265860 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.265929 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.266048 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.266121 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.266234 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.266314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.266385 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.266496 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.266569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.266680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.266754 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.266877 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.266990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.267101 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.267177 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.267289 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.267417 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.267505 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.267654 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.267737 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.267884 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.267961 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.268248 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.269085 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.269085 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.269221 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.269345 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.269459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.269554 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.269662 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.269748 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.269906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.269974 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.270153 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.270219 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.270454 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.270575 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.270674 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.270751 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.270853 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.270939 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.271088 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.271167 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.271324 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.271403 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.271562 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.271639 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.271798 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.271875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.272036 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] auth.external = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.272114 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.272269 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.272345 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.272519 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.272641 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.272797 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.272875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.273029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] auth.password = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.273106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.273262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] auth.token = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.273362 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.273500 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.273590 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.273736 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.273822 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] auth.external = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.273983 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.274061 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.274215 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.274301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.274463 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.274541 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.274709 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.274790 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] auth.password = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.274940 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.275025 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] auth.token = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.275176 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] credential.caching = True {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.275254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.275414 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.275489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.275660 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.275748 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.276147 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.276238 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.276386 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.276469 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.276652 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.276748 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.276890 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.276967 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.277132 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] credential.caching = True {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.277209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.277385 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.277472 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.277618 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.277695 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.277869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.277947 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.278108 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.278196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.278365 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.278456 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.278605 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.278684 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.278859 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.278948 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.279114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.279212 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.279349 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] federation.caching = True {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.279431 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.279584 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.279674 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.279828 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.279905 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.280074 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.280151 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.280305 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.280390 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.280548 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.280640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.280809 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.280887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.281062 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.281152 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] federation.caching = True {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.281298 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.281376 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.281549 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.281627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.281784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.281870 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.282028 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.282104 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.282259 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] identity.caching = True {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.282335 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.282506 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.282594 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.282750 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.282833 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.282985 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.283063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.283213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.283290 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.283452 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.283531 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.283731 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.284340 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.284441 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] identity.caching = True {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.284517 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.284623 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.284700 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.284700 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.284828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.284911 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.284985 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.285055 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.285125 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.285195 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.285264 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.285424 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.285504 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.285670 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.285749 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.285916 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.285995 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.286170 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.286260 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.286414 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.286493 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.286650 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.286726 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.286887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.286966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.287128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.287204 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.287357 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.287435 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.287593 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.287671 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.287823 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.287900 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.288056 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.288152 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.288297 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.288417 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.288537 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.288674 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.288807 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.288887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.289039 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.289124 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.289286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.289370 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.289521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.289603 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.289762 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.289841 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.290004 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.290091 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.290238 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.290319 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.290467 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.290551 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.290708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.290786 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.290939 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.291017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.291175 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.291263 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.291409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.291487 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.291646 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.291724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.291880 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.291960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.292115 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.292193 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.292355 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.292430 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.292609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.292709 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.292842 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.292928 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.293072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.293160 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.293314 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.293399 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.293556 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.293642 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.293787 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.293872 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.294021 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.294109 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.294255 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.294349 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.294484 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.user = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.294570 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.294719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.294804 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.294970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.295056 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.295196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.295289 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.295440 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.295527 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.295677 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.295763 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.295917 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.296011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.296147 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.296232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.user = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.296375 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.296461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.296669 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.296885 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.297014 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.297138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.297257 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.297390 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.297515 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.297647 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.297771 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.297894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.298017 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.298141 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.298266 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.298398 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.298398 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.298632 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.298756 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.298886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.299010 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.299155 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.299281 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.299432 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.299586 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.299646 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.299704 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.299770 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.299770 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.299873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.299930 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.299987 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.300198 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.300327 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.300460 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.300602 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.300724 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.300848 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.300972 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.301106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.301230 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.301357 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.301475 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.301598 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.301722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.301852 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.301977 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.302100 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.302233 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.302357 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.302481 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.302611 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.302735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.302858 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.302981 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.303104 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.303232 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.303364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.303488 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] resource.caching = True {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.303612 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.303734 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.303857 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.303990 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.304122 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.304245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.304245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.304478 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.304615 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.304734 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.304867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.304992 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.305115 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.305244 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.305367 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.305489 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] resource.caching = True {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.305620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.305753 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.305894 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] role.caching = True {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.305969 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.306038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] role.driver = sql {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.306106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.306252 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.306327 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.306396 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.306470 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.306540 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.306609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.306686 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.306757 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.306826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.306895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.307000 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.307079 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.307246 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.307324 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] role.caching = True {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.307489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.307566 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] role.driver = sql {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.307735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.307821 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.307967 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.308044 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.308211 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.308288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.308458 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.308543 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.308727 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.308807 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.308982 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.309059 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.309231 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.309397 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.309494 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.309636 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.309716 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.309885 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.309962 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.310135 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.310213 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.310382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.310459 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.310635 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.310713 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.310871 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.310950 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.311126 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.311289 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.311367 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.311533 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.311618 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.311786 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.311863 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.312038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.312114 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.312270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.312355 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.312520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.312617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.312796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.312872 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.313042 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.313123 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.313279 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.313355 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.313514 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.313590 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.313746 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.313823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.313981 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.314062 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] token.caching = True {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.314214 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.314289 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.314460 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.314538 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.314695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.314771 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.315489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.315637 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.315848 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.315918 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.316128 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.316258 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.316379 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.316498 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.316736 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] token.caching = True {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.316860 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.316985 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.317109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.317233 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.317357 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.317488 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.317622 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.317746 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.317870 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.317993 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.318115 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.318247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.318371 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.318495 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.318617 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.318740 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.318864 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.318995 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.319118 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.319252 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.319377 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] database.connection = **** {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.319509 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.319638 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.319763 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.319888 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.320018 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.320142 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.320266 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.320389 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.320512 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.320648 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.320780 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.320903 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.321032 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.321152 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.321272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] database.connection = **** {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.321404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.321532 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.321659 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.321780 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.321902 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.322021 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.322141 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.322261 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.322388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.322509 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.322628 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.322758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.322879 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.322998 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.323125 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.323267 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.323391 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.323467 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.323536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.323536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.323665 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.323736 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80478) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 05 19:48:29.323805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.323805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.323923 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.324097 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.324347 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.324596 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.324851 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.325091 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:29.325265 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80477) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 05 19:48:29.352975 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80478) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.353199 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80478) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.353421 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80478) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.353990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80477) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.354225 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80477) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.354448 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80477) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.362004 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80478) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.363130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80477) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.392913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80478) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.393732 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80478) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.394527 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80477) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.394720 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80477) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.447491 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80478) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.447977 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80477) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.498380 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80478) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.498644 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80478) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.499746 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80477) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.499995 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80477) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.564407 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80478) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.564684 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80478) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.564957 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80478) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.565193 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80478) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.565398 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80477) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.565585 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80478) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.565721 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80477) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.565846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80478) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.565987 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80477) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.566122 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80477) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.566244 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80477) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.566438 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80477) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.589094 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80478) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.589321 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80478) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.589504 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80477) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.589635 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80478) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.589760 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80477) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.589884 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80478) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.590017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80477) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.590157 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80478) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.590280 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80477) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.590439 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80478) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.590560 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80477) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.590681 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80478) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.590802 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80477) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.590802 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80477) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.601578 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80478) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.602173 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80477) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.614843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80478) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.615070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80478) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.615228 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80477) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.615365 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80478) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.615495 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80477) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.615621 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80478) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.615747 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80477) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.615882 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80478) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.616011 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80477) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.616135 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80478) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.616259 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80477) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.616383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80478) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.616517 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80477) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.616517 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80477) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:29.633346 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.633551 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 05 19:48:29.633882 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.634010 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.634148 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 05 19:48:29.634274 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 05 19:48:29.634397 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.634520 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.634643 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.634778 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 05 19:48:29.634914 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.635040 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.635163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.635285 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.635449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.635704 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.645566 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.645791 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.646054 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.646180 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.646331 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.646487 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.646649 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.646930 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.653493 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80478) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.654169 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80477) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.658191 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80478) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.658463 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.658834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.659063 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.659176 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80477) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.659365 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.659492 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.659617 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.659740 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.659862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.659994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.660120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.660243 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.660366 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.660490 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.660666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.660813 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.660937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.661067 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.661284 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.661572 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.661826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.684113 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.685350 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.685627 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80478) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.685898 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.686836 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80477) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.687119 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.691043 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80478) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.691314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.691591 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.692469 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80477) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.692775 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.693060 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.698283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80478) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.698559 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.699654 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80477) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.699944 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.702654 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80478) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.702935 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.703211 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.703495 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.703774 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.704054 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.704180 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80477) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.704334 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.704460 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.704612 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.704760 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.704951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.705239 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.705720 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.706022 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.706282 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.720470 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.720766 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.722334 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.722608 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.726135 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80478) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.726567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80477) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.729294 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.729588 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.729748 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.730044 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.734162 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80478) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.734463 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80477) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.738171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80478) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.738359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80477) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.741357 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.741565 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.741692 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.741866 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.741993 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.742162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.742290 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.742459 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.742585 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.742714 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.742851 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.743020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.758724 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80478) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.759006 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.759133 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80477) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.759258 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.759393 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.759518 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.759668 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.759929 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.766715 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.766939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.768767 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.769041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.769214 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.769498 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.771871 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80478) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.772147 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.772361 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80477) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.772487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.772669 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.772798 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.772949 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.773078 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.773207 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.773491 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.784682 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80478) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.784955 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.785228 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.785365 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80477) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.785492 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.785625 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.785761 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.785888 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.786013 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.786136 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.786260 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.786390 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.786613 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.786900 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.802293 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80478) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.803177 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80477) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.805279 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80478) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.806251 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80477) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.808202 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.809226 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.810731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.811721 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.812253 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80478) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.812535 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.812849 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.813276 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80477) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.813581 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.813880 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.819622 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.820415 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.821235 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80478) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.822059 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80477) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.824152 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.824437 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.824746 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.825021 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.825147 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.825375 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.825666 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.825940 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.833602 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80478) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.833894 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.834173 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.834583 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80477) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.834879 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.835168 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.841467 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80478) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.841738 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.842015 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.842153 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80477) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:29.842279 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.842402 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.842526 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.842649 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.842780 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.842906 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.843029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.843152 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.843275 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.843406 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.843633 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.843759 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.843892 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.844065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.844193 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.844317 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.844442 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.844568 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.844735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.844860 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.845012 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80478) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.845143 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.845336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.845612 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80477) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:29.878733 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55ca2d41c9c0 pid: 80478 (default app) Aug 05 19:48:29.882077 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe81153-36f2-444c-b58d-ad213da33c93 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:29.882365 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe81153-36f2-444c-b58d-ad213da33c93 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:29.882637 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe81153-36f2-444c-b58d-ad213da33c93 None None] PATH_INFO: `/v3` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:29.884018 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55ca2d41c9c0 pid: 80477 (default app) Aug 05 19:48:29.886881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1/1] 10.222.0.39 () {50 vars in 791 bytes} [Fri Aug 5 19:48:29 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:37.679757 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57068261-7e9b-4722-8c2a-bd63c922e938 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:37.680312 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57068261-7e9b-4722-8c2a-bd63c922e938 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:37.680312 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57068261-7e9b-4722-8c2a-bd63c922e938 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:37.684707 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1/2] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:48:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:48:37.690074 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab39e63-52e1-4394-9ec0-2fa577bd3e21 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:37.690577 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab39e63-52e1-4394-9ec0-2fa577bd3e21 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:37.690669 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab39e63-52e1-4394-9ec0-2fa577bd3e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:37.761656 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: INFO dbcounter [None req-1ab39e63-52e1-4394-9ec0-2fa577bd3e21 None None] Registered counter for database keystone Aug 05 19:48:37.782146 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG oslo_db.sqlalchemy.engines [None req-1ab39e63-52e1-4394-9ec0-2fa577bd3e21 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80478) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 05 19:48:37.789173 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writer thread running {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 05 19:48:37.967644 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG passlib.handlers.bcrypt [None req-1ab39e63-52e1-4394-9ec0-2fa577bd3e21 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80478) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 05 19:48:37.967967 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG passlib.handlers.bcrypt [None req-1ab39e63-52e1-4394-9ec0-2fa577bd3e21 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80478) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 05 19:48:38.023002 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-1ab39e63-52e1-4394-9ec0-2fa577bd3e21 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:38.024625 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-1ab39e63-52e1-4394-9ec0-2fa577bd3e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:38.157221 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 2/3] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 05 19:48:38.190860 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57068261-7e9b-4722-8c2a-bd63c922e938 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:38.260741 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: INFO dbcounter [None req-57068261-7e9b-4722-8c2a-bd63c922e938 None None] Registered counter for database keystone Aug 05 19:48:38.282304 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG oslo_db.sqlalchemy.engines [None req-57068261-7e9b-4722-8c2a-bd63c922e938 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80477) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 05 19:48:38.289284 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writer thread running {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 05 19:48:38.407493 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57068261-7e9b-4722-8c2a-bd63c922e938 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:38.412653 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aacda7-58fe-4e3a-847e-fbcb54d160b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:38.413308 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aacda7-58fe-4e3a-847e-fbcb54d160b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:38.413593 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aacda7-58fe-4e3a-847e-fbcb54d160b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:38.413868 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aacda7-58fe-4e3a-847e-fbcb54d160b0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:38.421900 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aacda7-58fe-4e3a-847e-fbcb54d160b0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:38.473875 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c7aacda7-58fe-4e3a-847e-fbcb54d160b0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 05 19:48:38.474218 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aacda7-58fe-4e3a-847e-fbcb54d160b0 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:38.483886 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 2/4] 10.222.0.39 () {58 vars in 1324 bytes} [Fri Aug 5 19:48:38 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 19:48:38.489963 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab39e63-52e1-4394-9ec0-2fa577bd3e21 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:38.504871 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab39e63-52e1-4394-9ec0-2fa577bd3e21 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:38.509316 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9baa97-6381-4be2-86e8-1274fe00bedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:38.510413 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9baa97-6381-4be2-86e8-1274fe00bedb admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:38.510695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9baa97-6381-4be2-86e8-1274fe00bedb admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:38.511013 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9baa97-6381-4be2-86e8-1274fe00bedb admin admin] PATH_INFO: `/v3/projects/9bb3822e6d1c4f82815dbeda0f6eff52` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:38.512429 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9baa97-6381-4be2-86e8-1274fe00bedb admin admin] RBAC: Authorizing `identity:get_project(project_id=9bb3822e6d1c4f82815dbeda0f6eff52)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:38.563797 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ac9baa97-6381-4be2-86e8-1274fe00bedb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 05 19:48:38.564139 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9baa97-6381-4be2-86e8-1274fe00bedb admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:38.569523 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 3/5] 10.222.0.39 () {56 vars in 1207 bytes} [Fri Aug 5 19:48:38 2022] GET /identity/v3/projects/9bb3822e6d1c4f82815dbeda0f6eff52 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:40.561130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb40739-7ae5-4ffb-bd9c-c55e3830f1ff None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:40.561608 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb40739-7ae5-4ffb-bd9c-c55e3830f1ff None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:40.561608 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb40739-7ae5-4ffb-bd9c-c55e3830f1ff None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:40.565798 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 3/6] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:48:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:40.571119 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815127ce-f054-4c63-a2d1-ba533f4a3ab6 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:40.571487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815127ce-f054-4c63-a2d1-ba533f4a3ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:40.571680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815127ce-f054-4c63-a2d1-ba533f4a3ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:40.591669 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-815127ce-f054-4c63-a2d1-ba533f4a3ab6 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:40.593207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-815127ce-f054-4c63-a2d1-ba533f4a3ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:40.602196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 4/7] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:40 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 05 19:48:40.624646 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb40739-7ae5-4ffb-bd9c-c55e3830f1ff None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:40.637210 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb40739-7ae5-4ffb-bd9c-c55e3830f1ff None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:40.641401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aa732b-a698-483f-bc45-38cf8973acbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:40.642008 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa732b-a698-483f-bc45-38cf8973acbd admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:40.642294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa732b-a698-483f-bc45-38cf8973acbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:40.642572 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa732b-a698-483f-bc45-38cf8973acbd admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:40.650661 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aa732b-a698-483f-bc45-38cf8973acbd admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:40.652710 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aa732b-a698-483f-bc45-38cf8973acbd admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:40.662327 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 4/8] 10.222.0.39 () {58 vars in 1324 bytes} [Fri Aug 5 19:48:40 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 19:48:40.667950 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815127ce-f054-4c63-a2d1-ba533f4a3ab6 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:40.677781 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815127ce-f054-4c63-a2d1-ba533f4a3ab6 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:40.682112 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b42c17-6efe-4887-89e4-28e5d91283a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:40.682732 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b42c17-6efe-4887-89e4-28e5d91283a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:40.683012 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b42c17-6efe-4887-89e4-28e5d91283a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:40.683283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b42c17-6efe-4887-89e4-28e5d91283a0 admin admin] PATH_INFO: `/v3/users/88c3879e2d344041935d28c7f22fdeeb` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:40.684654 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b42c17-6efe-4887-89e4-28e5d91283a0 admin admin] RBAC: Authorizing `identity:get_user(user_id=88c3879e2d344041935d28c7f22fdeeb)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:40.686462 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b42c17-6efe-4887-89e4-28e5d91283a0 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:40.687975 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 5/9] 10.222.0.39 () {56 vars in 1198 bytes} [Fri Aug 5 19:48:40 2022] GET /identity/v3/users/88c3879e2d344041935d28c7f22fdeeb => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:42.744947 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292a870-3862-4bc3-b51d-8c39e42e70fc None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:42.745490 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292a870-3862-4bc3-b51d-8c39e42e70fc None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:42.745490 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292a870-3862-4bc3-b51d-8c39e42e70fc None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:42.747506 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 5/10] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:48:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:42.753749 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c199ab-d779-4f34-a7a3-0767c5a473c2 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:42.754254 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c199ab-d779-4f34-a7a3-0767c5a473c2 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:42.754393 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c199ab-d779-4f34-a7a3-0767c5a473c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:42.774449 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-e8c199ab-d779-4f34-a7a3-0767c5a473c2 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:42.775637 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e8c199ab-d779-4f34-a7a3-0767c5a473c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:42.811286 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b419df-03fa-4e8a-ac10-a27b72313ef5 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:42.811670 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b419df-03fa-4e8a-ac10-a27b72313ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:42.812024 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b419df-03fa-4e8a-ac10-a27b72313ef5 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:42.813463 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 6/11] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:48:42.819483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cad0e-877b-44fe-a276-5802ca442a93 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:42.819774 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cad0e-877b-44fe-a276-5802ca442a93 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:42.820054 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cad0e-877b-44fe-a276-5802ca442a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:42.858481 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 6/12] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:42.863009 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG passlib.handlers.bcrypt [None req-7d9cad0e-877b-44fe-a276-5802ca442a93 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80477) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 05 19:48:42.863425 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG passlib.handlers.bcrypt [None req-7d9cad0e-877b-44fe-a276-5802ca442a93 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80477) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 05 19:48:42.881187 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c199ab-d779-4f34-a7a3-0767c5a473c2 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:42.904924 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:42.910359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-7d9cad0e-877b-44fe-a276-5802ca442a93 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:42.911958 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-7d9cad0e-877b-44fe-a276-5802ca442a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:42.987858 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c199ab-d779-4f34-a7a3-0767c5a473c2 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:43.015200 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 7/13] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:48:43.022673 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41264a-8015-4e48-9aab-9ca79f6d9de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:43.023386 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41264a-8015-4e48-9aab-9ca79f6d9de8 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:43.023733 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41264a-8015-4e48-9aab-9ca79f6d9de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:43.024012 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41264a-8015-4e48-9aab-9ca79f6d9de8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:43.035512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41264a-8015-4e48-9aab-9ca79f6d9de8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:48:43.038113 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9cad0e-877b-44fe-a276-5802ca442a93 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:43.044365 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-3c41264a-8015-4e48-9aab-9ca79f6d9de8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:48:43.044365 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:43.044365 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:43.044365 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:43.044365 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:43.044365 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:43.044365 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:43.044365 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:43.044365 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:43.044365 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:43.044365 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:43.044365 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:43.044365 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:48:43.044365 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:48:43.044365 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:48:43.044365 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:48:43.045321 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:48:43.045321 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:48:43.045321 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:48:43.045321 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:48:43.045321 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:43.045321 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:43.045321 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:43.045321 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:43.045321 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:43.045321 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:43.045321 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:43.045321 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:43.045321 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:43.045321 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:43.045321 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:43.045321 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:43.046140 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:43.046140 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:43.046140 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:43.046140 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:43.046140 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:48:43.046140 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:48:43.046140 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:48:43.046140 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:48:43.046140 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:48:43.046140 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:48:43.046140 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:48:43.046140 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:48:43.046140 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 7/14] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:48:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:48:43.052561 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41264a-8015-4e48-9aab-9ca79f6d9de8 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:43.136161 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:43.139550 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9cad0e-877b-44fe-a276-5802ca442a93 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:43.143892 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:43.156713 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41264a-8015-4e48-9aab-9ca79f6d9de8 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:43.180826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a77776-feec-48d7-a725-19fa4d6ea68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:43.181691 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a77776-feec-48d7-a725-19fa4d6ea68b None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:43.181981 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a77776-feec-48d7-a725-19fa4d6ea68b None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:43.182262 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a77776-feec-48d7-a725-19fa4d6ea68b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:43.192593 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13100a8e-67a8-422f-8937-0a1c59913a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:43.193226 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13100a8e-67a8-422f-8937-0a1c59913a8f None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:43.193510 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13100a8e-67a8-422f-8937-0a1c59913a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:43.193785 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13100a8e-67a8-422f-8937-0a1c59913a8f None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:43.194507 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13100a8e-67a8-422f-8937-0a1c59913a8f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:43.194815 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13100a8e-67a8-422f-8937-0a1c59913a8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:43.196212 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13100a8e-67a8-422f-8937-0a1c59913a8f None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:43.208588 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 8/15] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:48:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:48:43.214074 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13100a8e-67a8-422f-8937-0a1c59913a8f None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:43.251454 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a77776-feec-48d7-a725-19fa4d6ea68b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:43.253696 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a77776-feec-48d7-a725-19fa4d6ea68b None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:43.317535 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13100a8e-67a8-422f-8937-0a1c59913a8f None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:43.357324 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 8/16] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:48:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 19:48:43.364639 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a77776-feec-48d7-a725-19fa4d6ea68b None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:43.367853 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:43.458704 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd486e76-0e06-4a50-9bc6-c9211b18220d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:43.459338 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd486e76-0e06-4a50-9bc6-c9211b18220d None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:43.459618 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd486e76-0e06-4a50-9bc6-c9211b18220d None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:43.459892 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd486e76-0e06-4a50-9bc6-c9211b18220d None admin] PATH_INFO: `/v3/users/88c3879e2d344041935d28c7f22fdeeb` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:43.461076 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd486e76-0e06-4a50-9bc6-c9211b18220d None admin] RBAC: Authorizing `identity:get_user(user_id=88c3879e2d344041935d28c7f22fdeeb)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:43.462518 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd486e76-0e06-4a50-9bc6-c9211b18220d None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:43.464112 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 9/17] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:48:43 2022] GET /identity/v3/users/88c3879e2d344041935d28c7f22fdeeb => generated 247 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:48:43.469485 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a77776-feec-48d7-a725-19fa4d6ea68b None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:43.469693 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd486e76-0e06-4a50-9bc6-c9211b18220d None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:43.484215 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:43.506067 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aabe70-b34e-417c-8ff0-3c97430aebbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:43.506919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aabe70-b34e-417c-8ff0-3c97430aebbf None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:43.507195 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aabe70-b34e-417c-8ff0-3c97430aebbf None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:43.507467 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aabe70-b34e-417c-8ff0-3c97430aebbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:43.528275 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aabe70-b34e-417c-8ff0-3c97430aebbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:43.529763 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aabe70-b34e-417c-8ff0-3c97430aebbf None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:43.548561 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-11aabe70-b34e-417c-8ff0-3c97430aebbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:48:43.548561 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:43.548561 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:43.548561 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:43.548561 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:43.548561 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:43.548561 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:43.548561 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:43.548561 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:43.548561 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:43.548561 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:43.548561 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:43.548561 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:48:43.548561 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:48:43.548561 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:48:43.550294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:48:43.550294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:43.550294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:43.550294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:43.550294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:43.550294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:43.550294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:43.550294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:43.550294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:43.550294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:43.550294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:43.550294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:43.550294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:43.550294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:43.550294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:43.550294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:43.550294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:43.551294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:48:43.551294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:48:43.551294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:48:43.551294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:48:43.551294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 9/18] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:48:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:48:43.556600 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aabe70-b34e-417c-8ff0-3c97430aebbf None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:43.570410 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd486e76-0e06-4a50-9bc6-c9211b18220d None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:43.608676 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6917973e-d61e-4adc-b45a-fa0eef907c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:43.609318 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6917973e-d61e-4adc-b45a-fa0eef907c02 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:43.609603 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6917973e-d61e-4adc-b45a-fa0eef907c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:43.609877 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6917973e-d61e-4adc-b45a-fa0eef907c02 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:43.611042 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6917973e-d61e-4adc-b45a-fa0eef907c02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:43.613224 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6917973e-d61e-4adc-b45a-fa0eef907c02 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:43.614672 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:43.614820 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 10/19] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:48:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:48:43.620485 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6917973e-d61e-4adc-b45a-fa0eef907c02 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:43.660008 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aabe70-b34e-417c-8ff0-3c97430aebbf None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:43.693800 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea5a7a-710b-46aa-8828-cdf91818cfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:43.694410 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea5a7a-710b-46aa-8828-cdf91818cfa1 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:43.694693 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea5a7a-710b-46aa-8828-cdf91818cfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:43.694967 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea5a7a-710b-46aa-8828-cdf91818cfa1 None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:43.695662 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dea5a7a-710b-46aa-8828-cdf91818cfa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:43.695968 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dea5a7a-710b-46aa-8828-cdf91818cfa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:43.697388 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dea5a7a-710b-46aa-8828-cdf91818cfa1 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:43.701545 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:43.715642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 10/20] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:48:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:48:43.722526 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6917973e-d61e-4adc-b45a-fa0eef907c02 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:43.761574 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd1b83-adfe-44ce-a216-78e067928258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:43.762255 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd1b83-adfe-44ce-a216-78e067928258 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:43.762467 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd1b83-adfe-44ce-a216-78e067928258 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:43.762740 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd1b83-adfe-44ce-a216-78e067928258 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:43.763559 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd1b83-adfe-44ce-a216-78e067928258 None admin] RBAC: Adding query filter params (role.id=89919f784ad144529b07c8ab92ee7d48, scope.domain.id=default, user.id=88c3879e2d344041935d28c7f22fdeeb) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:43.763872 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd1b83-adfe-44ce-a216-78e067928258 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:43.765417 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd1b83-adfe-44ce-a216-78e067928258 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:43.774264 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 11/21] 10.222.0.39 () {56 vars in 1322 bytes} [Fri Aug 5 19:48:43 2022] GET /identity/v3/role_assignments?user.id=88c3879e2d344041935d28c7f22fdeeb&scope.domain.id=default&role.id=89919f784ad144529b07c8ab92ee7d48 => generated 233 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:48:46.363106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f8760a-2453-431a-be63-b75a2416d451 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:46.363955 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f8760a-2453-431a-be63-b75a2416d451 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:46.364393 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f8760a-2453-431a-be63-b75a2416d451 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:46.365918 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 11/22] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:48:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:46.371526 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7afb3c-8bfa-46b4-a669-7bec236a5ef9 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:46.372273 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7afb3c-8bfa-46b4-a669-7bec236a5ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:46.372711 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7afb3c-8bfa-46b4-a669-7bec236a5ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:46.402269 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-7f7afb3c-8bfa-46b4-a669-7bec236a5ef9 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:46.403687 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-7f7afb3c-8bfa-46b4-a669-7bec236a5ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:46.407008 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273fa12-6c61-47ae-aec6-3befe13e0c7d None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:46.407444 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273fa12-6c61-47ae-aec6-3befe13e0c7d None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:46.407878 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273fa12-6c61-47ae-aec6-3befe13e0c7d None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:46.409460 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 12/23] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:48:46.414717 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0731b-187f-4b42-8a6a-11b8621f409e None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:46.415164 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0731b-187f-4b42-8a6a-11b8621f409e None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:46.415577 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0731b-187f-4b42-8a6a-11b8621f409e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:46.445461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-eac0731b-187f-4b42-8a6a-11b8621f409e None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:46.448333 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-eac0731b-187f-4b42-8a6a-11b8621f409e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:46.586107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 13/24] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:48:46.586510 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 12/25] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:48:46.616984 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259e927-3447-4941-bea9-58a94c69fbcc None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:46.617267 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259e927-3447-4941-bea9-58a94c69fbcc None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:46.617727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259e927-3447-4941-bea9-58a94c69fbcc None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:46.619440 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 14/26] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:46.624258 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7afb3c-8bfa-46b4-a669-7bec236a5ef9 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:46.624743 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157ab6c-f491-4559-bb52-01925348503b None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:46.625038 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157ab6c-f491-4559-bb52-01925348503b None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:46.625316 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157ab6c-f491-4559-bb52-01925348503b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:46.651263 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-f157ab6c-f491-4559-bb52-01925348503b None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:46.652548 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-f157ab6c-f491-4559-bb52-01925348503b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:46.726238 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:46.729982 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:46.757817 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7afb3c-8bfa-46b4-a669-7bec236a5ef9 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:46.763009 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 15/27] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:48:46.772599 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f157ab6c-f491-4559-bb52-01925348503b None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:46.802082 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15e3cfb-266d-4caf-b9ac-7042f47ca583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:46.803132 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e3cfb-266d-4caf-b9ac-7042f47ca583 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:46.803607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e3cfb-266d-4caf-b9ac-7042f47ca583 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:46.804068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e3cfb-266d-4caf-b9ac-7042f47ca583 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:46.815817 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15e3cfb-266d-4caf-b9ac-7042f47ca583 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:46.817706 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15e3cfb-266d-4caf-b9ac-7042f47ca583 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:46.835409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-a15e3cfb-266d-4caf-b9ac-7042f47ca583 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:48:46.835409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:46.835409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:46.835409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:46.835409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:46.835409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:46.835409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:46.835409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:46.835409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:46.835409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:46.835409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:46.835409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:46.835409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:48:46.835409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:48:46.835409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:48:46.836271 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:48:46.836271 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:46.836271 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:46.836271 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:46.836271 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:46.836271 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:46.836271 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:46.836271 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:46.836271 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:46.836271 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:46.836271 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:46.836271 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:46.836271 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:46.836271 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:46.836271 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:46.836271 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:46.836271 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:46.837086 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:48:46.837086 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:48:46.837086 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:48:46.837086 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:48:46.839040 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 13/28] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:48:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:48:46.846686 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbafe6-bdd6-4946-8234-edd8b1885321 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:46.847168 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbafe6-bdd6-4946-8234-edd8b1885321 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:46.847584 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbafe6-bdd6-4946-8234-edd8b1885321 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:46.849441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 14/29] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:48:46 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:46.850502 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fbafe6-bdd6-4946-8234-edd8b1885321 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:46.895513 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f157ab6c-f491-4559-bb52-01925348503b None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:46.933578 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ddd06c-7131-4be9-bb7b-779df47186f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:46.934200 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ddd06c-7131-4be9-bb7b-779df47186f0 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:46.934758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ddd06c-7131-4be9-bb7b-779df47186f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:46.935032 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ddd06c-7131-4be9-bb7b-779df47186f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:46.947889 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ddd06c-7131-4be9-bb7b-779df47186f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:48:46.957640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-48ddd06c-7131-4be9-bb7b-779df47186f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:48:46.957640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:46.957640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:46.957640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:46.957640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:46.957640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:46.957640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:46.957640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:46.957640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:46.957640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:46.957640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:46.957640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:46.957640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:48:46.957640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:48:46.957640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:48:46.957640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:48:46.958558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:48:46.958558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:48:46.958558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:48:46.958558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:48:46.958558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:46.958558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:46.958558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:46.958558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:46.958558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:46.958558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:46.958558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:46.958558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:46.958558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:46.958558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:46.958558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:46.958558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:46.959296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:46.959296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:46.959296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:46.959296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:46.959296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:48:46.959296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:48:46.959296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:48:46.959296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:48:46.959296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:48:46.959296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:48:46.959296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:48:46.959296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:48:46.961183 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fbafe6-bdd6-4946-8234-edd8b1885321 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:46.961937 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 16/30] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:48:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:48:46.962930 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ddd06c-7131-4be9-bb7b-779df47186f0 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:46.992875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:47.003698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86cf11d-73d4-45b3-a512-a3607e277c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:47.004496 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86cf11d-73d4-45b3-a512-a3607e277c15 None admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:47.004812 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86cf11d-73d4-45b3-a512-a3607e277c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:47.005145 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86cf11d-73d4-45b3-a512-a3607e277c15 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:47.005881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86cf11d-73d4-45b3-a512-a3607e277c15 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:47.007362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86cf11d-73d4-45b3-a512-a3607e277c15 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:47.115910 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:47.166477 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 15/31] 10.222.0.39 () {60 vars in 1131 bytes} [Fri Aug 5 19:48:46 2022] POST /identity/v3/roles => generated 219 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:48:47.173028 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15cd154-c005-4680-bd27-d2c517464a71 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:47.173878 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15cd154-c005-4680-bd27-d2c517464a71 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:47.174446 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15cd154-c005-4680-bd27-d2c517464a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:47.187934 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ddd06c-7131-4be9-bb7b-779df47186f0 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:47.197317 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-c15cd154-c005-4680-bd27-d2c517464a71 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:47.199493 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-c15cd154-c005-4680-bd27-d2c517464a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:47.230746 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd0403c-5f2a-436b-98c4-c05b5eded450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:47.231682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0403c-5f2a-436b-98c4-c05b5eded450 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:47.232113 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0403c-5f2a-436b-98c4-c05b5eded450 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:47.232535 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0403c-5f2a-436b-98c4-c05b5eded450 None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:47.233528 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd0403c-5f2a-436b-98c4-c05b5eded450 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:47.234085 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd0403c-5f2a-436b-98c4-c05b5eded450 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:47.235755 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd0403c-5f2a-436b-98c4-c05b5eded450 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:47.250822 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 17/32] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:48:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:47.252139 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd0403c-5f2a-436b-98c4-c05b5eded450 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:47.284559 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 16/33] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:47.286167 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59cbc3c-08c7-4325-ba6b-7f427440572b None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:47.286744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59cbc3c-08c7-4325-ba6b-7f427440572b None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:47.287236 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59cbc3c-08c7-4325-ba6b-7f427440572b None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:47.288890 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 17/34] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:48:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:47.289957 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59cbc3c-08c7-4325-ba6b-7f427440572b None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:47.369838 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:47.378014 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd0403c-5f2a-436b-98c4-c05b5eded450 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:47.426845 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59cbc3c-08c7-4325-ba6b-7f427440572b None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:47.443335 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ffb1c4-cbe6-4587-b9e8-87f331cdb355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:47.443991 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ffb1c4-cbe6-4587-b9e8-87f331cdb355 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:47.444275 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ffb1c4-cbe6-4587-b9e8-87f331cdb355 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:47.444623 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ffb1c4-cbe6-4587-b9e8-87f331cdb355 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:47.445564 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ffb1c4-cbe6-4587-b9e8-87f331cdb355 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:47.445871 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ffb1c4-cbe6-4587-b9e8-87f331cdb355 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:47.447250 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ffb1c4-cbe6-4587-b9e8-87f331cdb355 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:47.452295 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:47.461512 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 18/35] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:48:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:47.462587 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97d748-2a33-4898-9624-227de4f4f12d None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:47.462883 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97d748-2a33-4898-9624-227de4f4f12d None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:47.463147 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97d748-2a33-4898-9624-227de4f4f12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:47.477385 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58eda49-7166-4a03-84ed-d380187e336b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:47.478008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58eda49-7166-4a03-84ed-d380187e336b None admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:47.478490 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58eda49-7166-4a03-84ed-d380187e336b None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:47.478777 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58eda49-7166-4a03-84ed-d380187e336b None admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:47.479468 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58eda49-7166-4a03-84ed-d380187e336b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:47.481091 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58eda49-7166-4a03-84ed-d380187e336b None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:47.481951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-ca97d748-2a33-4898-9624-227de4f4f12d None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:47.483153 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-ca97d748-2a33-4898-9624-227de4f4f12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:47.513776 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 18/36] 10.222.0.39 () {60 vars in 1141 bytes} [Fri Aug 5 19:48:47 2022] POST /identity/v3/projects => generated 301 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:48:47.514698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58eda49-7166-4a03-84ed-d380187e336b None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:47.574655 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 19/37] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:47.575916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca97d748-2a33-4898-9624-227de4f4f12d None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:47.670392 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58eda49-7166-4a03-84ed-d380187e336b None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:47.701092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:47.715795 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca97d748-2a33-4898-9624-227de4f4f12d None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:47.716547 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86500e-dd27-41aa-acc1-8c778a20fde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:47.717239 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86500e-dd27-41aa-acc1-8c778a20fde9 None admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:47.717666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86500e-dd27-41aa-acc1-8c778a20fde9 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:47.717955 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86500e-dd27-41aa-acc1-8c778a20fde9 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:47.719527 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86500e-dd27-41aa-acc1-8c778a20fde9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:47.721984 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86500e-dd27-41aa-acc1-8c778a20fde9 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:47.739777 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 19/38] 10.222.0.39 () {60 vars in 1131 bytes} [Fri Aug 5 19:48:47 2022] POST /identity/v3/roles => generated 225 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:48:47.741125 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86500e-dd27-41aa-acc1-8c778a20fde9 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:47.757224 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ee9f0-0e56-4776-90de-ead62eec76a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:47.757900 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ee9f0-0e56-4776-90de-ead62eec76a2 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:47.758179 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ee9f0-0e56-4776-90de-ead62eec76a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:47.758808 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ee9f0-0e56-4776-90de-ead62eec76a2 None admin] PATH_INFO: `/v3/users/88c3879e2d344041935d28c7f22fdeeb` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:47.760472 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ee9f0-0e56-4776-90de-ead62eec76a2 None admin] RBAC: Authorizing `identity:get_user(user_id=88c3879e2d344041935d28c7f22fdeeb)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:47.762012 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ee9f0-0e56-4776-90de-ead62eec76a2 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:47.768683 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 20/39] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:48:47 2022] GET /identity/v3/users/88c3879e2d344041935d28c7f22fdeeb => generated 247 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:47.768683 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35d2ea-5b38-48fe-9b84-7198c3a86ffa None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:47.768683 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35d2ea-5b38-48fe-9b84-7198c3a86ffa None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:47.768683 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35d2ea-5b38-48fe-9b84-7198c3a86ffa None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:47.768683 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 21/40] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:47.774301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a35d2ea-5b38-48fe-9b84-7198c3a86ffa None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:47.787293 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:47.875944 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86500e-dd27-41aa-acc1-8c778a20fde9 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:47.903749 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a35d2ea-5b38-48fe-9b84-7198c3a86ffa None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:47.925164 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9938506-83c4-4990-be2f-0bb17cc75e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:47.925809 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9938506-83c4-4990-be2f-0bb17cc75e3e None admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:47.926096 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9938506-83c4-4990-be2f-0bb17cc75e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:47.926376 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9938506-83c4-4990-be2f-0bb17cc75e3e None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:47.927111 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9938506-83c4-4990-be2f-0bb17cc75e3e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:47.928500 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9938506-83c4-4990-be2f-0bb17cc75e3e None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:47.948466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 20/41] 10.222.0.39 () {60 vars in 1131 bytes} [Fri Aug 5 19:48:47 2022] POST /identity/v3/roles => generated 223 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:48:47.950644 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097958f7-6071-474a-bbf7-e94854c0ee0d None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:47.950644 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097958f7-6071-474a-bbf7-e94854c0ee0d None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:47.950644 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097958f7-6071-474a-bbf7-e94854c0ee0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:47.953124 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ae5df-d334-4b7a-8ddf-1078c86ebb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:47.953698 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ae5df-d334-4b7a-8ddf-1078c86ebb22 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:47.954049 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ae5df-d334-4b7a-8ddf-1078c86ebb22 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:47.954331 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ae5df-d334-4b7a-8ddf-1078c86ebb22 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:47.960000 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ae5df-d334-4b7a-8ddf-1078c86ebb22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:47.960000 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ae5df-d334-4b7a-8ddf-1078c86ebb22 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:47.960762 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 22/42] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:48:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:47.969612 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ae5df-d334-4b7a-8ddf-1078c86ebb22 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:47.971484 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-097958f7-6071-474a-bbf7-e94854c0ee0d None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:47.972780 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-097958f7-6071-474a-bbf7-e94854c0ee0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:48.000251 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:48.057402 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 21/43] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:48.082110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097958f7-6071-474a-bbf7-e94854c0ee0d None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:48.090511 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ae5df-d334-4b7a-8ddf-1078c86ebb22 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:48.104605 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:48.145975 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a5513-27e9-4b5c-94a3-c9052413874d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:48.146423 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a5513-27e9-4b5c-94a3-c9052413874d None admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:48.146774 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a5513-27e9-4b5c-94a3-c9052413874d None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:48.147044 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a5513-27e9-4b5c-94a3-c9052413874d None admin] PATH_INFO: `/v3/domains/default/users/88c3879e2d344041935d28c7f22fdeeb/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:48.149151 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1a5513-27e9-4b5c-94a3-c9052413874d None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=88c3879e2d344041935d28c7f22fdeeb, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:48.152389 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1a5513-27e9-4b5c-94a3-c9052413874d None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:48.205525 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097958f7-6071-474a-bbf7-e94854c0ee0d None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:48.239415 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc0fc6-0567-4130-a58e-9effe99806e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:48.240052 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc0fc6-0567-4130-a58e-9effe99806e8 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:48.240335 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc0fc6-0567-4130-a58e-9effe99806e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:48.240645 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc0fc6-0567-4130-a58e-9effe99806e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:48.241789 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc0fc6-0567-4130-a58e-9effe99806e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:48.243234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc0fc6-0567-4130-a58e-9effe99806e8 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:48.244837 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 22/44] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:48:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:48:48.250754 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc0fc6-0567-4130-a58e-9effe99806e8 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:48.273647 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 23/45] 10.222.0.39 () {58 vars in 1361 bytes} [Fri Aug 5 19:48:47 2022] PUT /identity/v3/domains/default/users/88c3879e2d344041935d28c7f22fdeeb/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:48:48.343230 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:48.360131 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc0fc6-0567-4130-a58e-9effe99806e8 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:48.395698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c30301f-8df6-4fb0-a783-54621c88b5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:48.396327 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30301f-8df6-4fb0-a783-54621c88b5f0 None admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:48.396642 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30301f-8df6-4fb0-a783-54621c88b5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:48.396920 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30301f-8df6-4fb0-a783-54621c88b5f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:48.397614 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c30301f-8df6-4fb0-a783-54621c88b5f0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:48.399070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c30301f-8df6-4fb0-a783-54621c88b5f0 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:48.430428 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 23/46] 10.222.0.39 () {60 vars in 1141 bytes} [Fri Aug 5 19:48:48 2022] POST /identity/v3/projects => generated 312 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:48:50.263480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecaff18-934f-463c-a04d-bc6f9f98ec29 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:50.264010 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecaff18-934f-463c-a04d-bc6f9f98ec29 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:50.264093 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecaff18-934f-463c-a04d-bc6f9f98ec29 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:50.269238 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 24/47] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:48:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:50.274531 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86e3c3-90c0-49ed-b9b8-0d69c9cab127 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:50.274957 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86e3c3-90c0-49ed-b9b8-0d69c9cab127 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:50.275100 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86e3c3-90c0-49ed-b9b8-0d69c9cab127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:50.293153 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-8d86e3c3-90c0-49ed-b9b8-0d69c9cab127 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:50.294341 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-8d86e3c3-90c0-49ed-b9b8-0d69c9cab127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:50.365397 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 24/48] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:50.387932 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecaff18-934f-463c-a04d-bc6f9f98ec29 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:50.447380 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a66c2-40b5-45cd-b0cd-72822cda53de None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:50.447585 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a66c2-40b5-45cd-b0cd-72822cda53de None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:50.447857 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a66c2-40b5-45cd-b0cd-72822cda53de None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:50.449398 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 25/49] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:48:50.454008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322be4da-9ce2-4d61-b547-f209fd2da112 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:50.454289 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322be4da-9ce2-4d61-b547-f209fd2da112 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:50.455139 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322be4da-9ce2-4d61-b547-f209fd2da112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:50.472943 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-322be4da-9ce2-4d61-b547-f209fd2da112 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:50.474112 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-322be4da-9ce2-4d61-b547-f209fd2da112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:50.481143 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:50.500334 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecaff18-934f-463c-a04d-bc6f9f98ec29 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:50.535459 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5c6b3-5c0c-426c-be81-49fad9acd480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:50.536102 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5c6b3-5c0c-426c-be81-49fad9acd480 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:50.536389 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5c6b3-5c0c-426c-be81-49fad9acd480 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:50.536645 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:50.536740 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5c6b3-5c0c-426c-be81-49fad9acd480 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:50.544000 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 26/50] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:50.547986 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea5c6b3-5c0c-426c-be81-49fad9acd480 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:48:50.553711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-3ea5c6b3-5c0c-426c-be81-49fad9acd480 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:48:50.553711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:50.553711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:50.553711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:50.553711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:50.553711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:50.553711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:50.553711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:50.553711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:50.553711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:50.553711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:50.553711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:50.553711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:48:50.553711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:48:50.553711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:48:50.553711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:48:50.554681 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:48:50.554681 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:48:50.554681 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:48:50.554681 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:48:50.554681 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:50.554681 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:50.554681 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:50.554681 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:50.554681 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:50.554681 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:50.554681 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:50.554681 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:50.554681 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:50.554681 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:50.554681 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:50.554681 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:50.555642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:50.555642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:50.555642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:50.555642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:50.555642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:48:50.555642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:48:50.555642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:48:50.555642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:48:50.555642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:48:50.555642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:48:50.555642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:48:50.555642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:48:50.555642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 25/51] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:48:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:48:50.562524 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322be4da-9ce2-4d61-b547-f209fd2da112 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:50.567711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5c6b3-5c0c-426c-be81-49fad9acd480 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:50.666648 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322be4da-9ce2-4d61-b547-f209fd2da112 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:50.669495 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5c6b3-5c0c-426c-be81-49fad9acd480 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:50.705863 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b07dc4-8942-44a4-8a24-c17635be4448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:50.706055 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a08bef-b52d-4ccf-94f9-349f9f9535fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:50.706158 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b07dc4-8942-44a4-8a24-c17635be4448 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:50.706158 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b07dc4-8942-44a4-8a24-c17635be4448 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:50.706339 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a08bef-b52d-4ccf-94f9-349f9f9535fb None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:50.706420 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b07dc4-8942-44a4-8a24-c17635be4448 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:50.706497 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a08bef-b52d-4ccf-94f9-349f9f9535fb None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:50.706497 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a08bef-b52d-4ccf-94f9-349f9f9535fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:50.706745 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b07dc4-8942-44a4-8a24-c17635be4448 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:50.707058 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b07dc4-8942-44a4-8a24-c17635be4448 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:50.707707 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a08bef-b52d-4ccf-94f9-349f9f9535fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:50.708454 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b07dc4-8942-44a4-8a24-c17635be4448 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:50.709293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a08bef-b52d-4ccf-94f9-349f9f9535fb None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:50.710877 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 26/52] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:48:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:48:50.717003 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a08bef-b52d-4ccf-94f9-349f9f9535fb None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:50.718772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 27/53] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:48:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:50.724903 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b07dc4-8942-44a4-8a24-c17635be4448 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:50.731795 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:50.784760 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:50.826148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a08bef-b52d-4ccf-94f9-349f9f9535fb None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:50.828273 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b07dc4-8942-44a4-8a24-c17635be4448 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:50.862216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cfe97b-cabf-4849-ae8b-cdff32ad847e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:50.862697 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5cd985-2985-4f4b-bdeb-b5bf4492a1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:50.862835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cfe97b-cabf-4849-ae8b-cdff32ad847e None admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:50.863123 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cfe97b-cabf-4849-ae8b-cdff32ad847e None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:50.863334 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cd985-2985-4f4b-bdeb-b5bf4492a1de None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:50.863419 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cfe97b-cabf-4849-ae8b-cdff32ad847e None admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:50.863618 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cd985-2985-4f4b-bdeb-b5bf4492a1de None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:50.863894 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cd985-2985-4f4b-bdeb-b5bf4492a1de None admin] PATH_INFO: `/v3/users/88c3879e2d344041935d28c7f22fdeeb` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:50.864078 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cfe97b-cabf-4849-ae8b-cdff32ad847e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:50.865100 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5cd985-2985-4f4b-bdeb-b5bf4492a1de None admin] RBAC: Authorizing `identity:get_user(user_id=88c3879e2d344041935d28c7f22fdeeb)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:50.865567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cfe97b-cabf-4849-ae8b-cdff32ad847e None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:50.866518 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5cd985-2985-4f4b-bdeb-b5bf4492a1de None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:50.868108 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 28/54] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:48:50 2022] GET /identity/v3/users/88c3879e2d344041935d28c7f22fdeeb => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:48:50.873476 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5cd985-2985-4f4b-bdeb-b5bf4492a1de None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:50.896612 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 27/55] 10.222.0.39 () {60 vars in 1140 bytes} [Fri Aug 5 19:48:50 2022] POST /identity/v3/projects => generated 298 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:48:50.983246 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5cd985-2985-4f4b-bdeb-b5bf4492a1de None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:51.011018 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:51.019555 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89fd17-a734-46ff-8eb0-1db1e0913858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:51.020200 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89fd17-a734-46ff-8eb0-1db1e0913858 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:51.020480 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89fd17-a734-46ff-8eb0-1db1e0913858 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:51.020782 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89fd17-a734-46ff-8eb0-1db1e0913858 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:51.021981 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae89fd17-a734-46ff-8eb0-1db1e0913858 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:51.023430 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae89fd17-a734-46ff-8eb0-1db1e0913858 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:51.025089 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 29/56] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:48:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:48:51.031291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cfe97b-cabf-4849-ae8b-cdff32ad847e None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:51.094745 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:51.136944 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cfe97b-cabf-4849-ae8b-cdff32ad847e None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:51.171604 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c5654-97f7-4093-9ca9-847de96fbdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:51.172218 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c5654-97f7-4093-9ca9-847de96fbdcb None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:51.172504 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c5654-97f7-4093-9ca9-847de96fbdcb None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:51.172819 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c5654-97f7-4093-9ca9-847de96fbdcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:51.174324 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c5654-97f7-4093-9ca9-847de96fbdcb None admin] RBAC: Adding query filter params (role.id=89919f784ad144529b07c8ab92ee7d48, scope.domain.id=default, user.id=88c3879e2d344041935d28c7f22fdeeb) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:51.174644 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c5654-97f7-4093-9ca9-847de96fbdcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:51.176144 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c5654-97f7-4093-9ca9-847de96fbdcb None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:51.184450 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 28/57] 10.222.0.39 () {56 vars in 1322 bytes} [Fri Aug 5 19:48:51 2022] GET /identity/v3/role_assignments?user.id=88c3879e2d344041935d28c7f22fdeeb&scope.domain.id=default&role.id=89919f784ad144529b07c8ab92ee7d48 => generated 530 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:52.922830 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84298d0-988f-4964-9bf8-a52a42242bbd None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:52.923341 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84298d0-988f-4964-9bf8-a52a42242bbd None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:52.923453 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84298d0-988f-4964-9bf8-a52a42242bbd None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:52.924974 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 30/58] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:48:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:52.930033 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804141db-9c88-438d-924f-425efc68f365 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:52.930396 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804141db-9c88-438d-924f-425efc68f365 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:52.930584 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804141db-9c88-438d-924f-425efc68f365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:52.948371 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-804141db-9c88-438d-924f-425efc68f365 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:52.949900 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-804141db-9c88-438d-924f-425efc68f365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:53.019627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 29/59] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:53.044624 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84298d0-988f-4964-9bf8-a52a42242bbd None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:53.146593 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84298d0-988f-4964-9bf8-a52a42242bbd None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:53.180261 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89450ee1-2035-4429-84c8-399f7f315faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:53.180854 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89450ee1-2035-4429-84c8-399f7f315faa None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:53.181135 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89450ee1-2035-4429-84c8-399f7f315faa None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:53.181406 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89450ee1-2035-4429-84c8-399f7f315faa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:53.182541 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89450ee1-2035-4429-84c8-399f7f315faa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:53.183977 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89450ee1-2035-4429-84c8-399f7f315faa None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:53.185579 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 31/60] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:48:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:53.191796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804141db-9c88-438d-924f-425efc68f365 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:53.248123 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:53.289697 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804141db-9c88-438d-924f-425efc68f365 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:53.322651 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25c927-62fd-4587-b27a-7e13c9f91cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:53.323252 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25c927-62fd-4587-b27a-7e13c9f91cc6 None admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:53.323530 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25c927-62fd-4587-b27a-7e13c9f91cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:53.323803 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25c927-62fd-4587-b27a-7e13c9f91cc6 None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:53.324635 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25c927-62fd-4587-b27a-7e13c9f91cc6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:53.326064 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25c927-62fd-4587-b27a-7e13c9f91cc6 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:53.353046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 30/61] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Aug 5 19:48:53 2022] POST /identity/v3/users => generated 275 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:48:56.391559 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6853c7-5711-4654-9285-0f361e84e825 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:56.392889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6853c7-5711-4654-9285-0f361e84e825 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:56.393393 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6853c7-5711-4654-9285-0f361e84e825 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:56.395013 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 32/62] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:48:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:56.400799 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14480ed-19dd-4779-941b-48ba5ad2909f None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:56.401642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14480ed-19dd-4779-941b-48ba5ad2909f None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:56.402146 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14480ed-19dd-4779-941b-48ba5ad2909f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:56.429613 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-c14480ed-19dd-4779-941b-48ba5ad2909f None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:56.431184 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-c14480ed-19dd-4779-941b-48ba5ad2909f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:56.522898 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 31/63] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:56.559500 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6853c7-5711-4654-9285-0f361e84e825 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:56.643441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:56.702313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6853c7-5711-4654-9285-0f361e84e825 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:56.726744 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eebd65-16d4-42f5-8a42-f7e58623952e None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:56.727377 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eebd65-16d4-42f5-8a42-f7e58623952e None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:56.727865 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eebd65-16d4-42f5-8a42-f7e58623952e None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:56.729593 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 32/64] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:48:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:56.736832 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c069213-f7fd-4975-a0a3-d75e9271df61 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:56.737424 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c069213-f7fd-4975-a0a3-d75e9271df61 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:56.737890 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c069213-f7fd-4975-a0a3-d75e9271df61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:56.753344 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db8dc6-1f29-44a0-a1f5-8004ce98332c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:56.754000 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db8dc6-1f29-44a0-a1f5-8004ce98332c None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:56.754290 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db8dc6-1f29-44a0-a1f5-8004ce98332c None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:56.754573 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db8dc6-1f29-44a0-a1f5-8004ce98332c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:56.757529 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-9c069213-f7fd-4975-a0a3-d75e9271df61 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:56.758905 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-9c069213-f7fd-4975-a0a3-d75e9271df61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:56.777485 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db8dc6-1f29-44a0-a1f5-8004ce98332c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:48:56.783187 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-c6db8dc6-1f29-44a0-a1f5-8004ce98332c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:48:56.783187 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:56.783187 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:56.783187 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:56.783187 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:56.783187 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:56.783187 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:56.783187 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:56.783187 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:56.783187 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:56.783187 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:56.783187 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:56.783187 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:48:56.783187 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:48:56.783187 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:48:56.783187 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:48:56.784050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:48:56.784050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:48:56.784050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:48:56.784050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:48:56.784050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:56.784050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:56.784050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:56.784050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:56.784050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:56.784050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:56.784050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:56.784050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:56.784050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:56.784050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:56.784050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:56.784050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:56.784832 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:56.784832 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:56.784832 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:56.784832 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:56.784832 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:48:56.784832 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:48:56.784832 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:48:56.784832 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:48:56.784832 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:48:56.784832 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:48:56.784832 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:48:56.784832 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:48:56.784832 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 33/65] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:48:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:48:56.792036 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db8dc6-1f29-44a0-a1f5-8004ce98332c None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:56.809534 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:56.842387 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 33/66] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:48:56.885784 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c069213-f7fd-4975-a0a3-d75e9271df61 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:56.968409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db8dc6-1f29-44a0-a1f5-8004ce98332c None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:57.006339 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:57.027319 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b29d1-1def-4adc-8510-d6d3da9353f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:57.028474 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b29d1-1def-4adc-8510-d6d3da9353f3 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:57.028793 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b29d1-1def-4adc-8510-d6d3da9353f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:57.029070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b29d1-1def-4adc-8510-d6d3da9353f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:57.030085 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b29d1-1def-4adc-8510-d6d3da9353f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:57.030397 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b29d1-1def-4adc-8510-d6d3da9353f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:57.032446 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b29d1-1def-4adc-8510-d6d3da9353f3 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:57.044662 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 34/67] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Aug 5 19:48:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:48:57.046380 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5035bf3-40c5-417c-b9a8-d860cb4908c0 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:57.047189 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5035bf3-40c5-417c-b9a8-d860cb4908c0 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:57.047866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5035bf3-40c5-417c-b9a8-d860cb4908c0 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:57.049941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 35/68] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:48:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:57.050708 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c069213-f7fd-4975-a0a3-d75e9271df61 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:57.053029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5035bf3-40c5-417c-b9a8-d860cb4908c0 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:57.109565 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafcbd8f-6edf-400b-bd02-ec823f27b964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:57.110742 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafcbd8f-6edf-400b-bd02-ec823f27b964 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:57.111286 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafcbd8f-6edf-400b-bd02-ec823f27b964 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:57.111705 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafcbd8f-6edf-400b-bd02-ec823f27b964 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:57.126858 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafcbd8f-6edf-400b-bd02-ec823f27b964 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:48:57.133617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-cafcbd8f-6edf-400b-bd02-ec823f27b964 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:48:57.133617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:57.133617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:57.133617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:57.133617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:57.133617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:57.133617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:57.133617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:57.133617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:57.133617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:57.133617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:57.133617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:57.133617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:48:57.133617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:48:57.133617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:48:57.133617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:48:57.134707 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:48:57.134707 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:48:57.134707 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:48:57.134707 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:48:57.134707 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:57.134707 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:57.134707 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:57.134707 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:57.134707 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:57.134707 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:57.134707 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:57.134707 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:57.134707 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:57.134707 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:57.134707 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:57.134707 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:57.135714 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:57.135714 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:57.135714 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:57.135714 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:57.135714 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:48:57.135714 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:48:57.135714 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:48:57.135714 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:48:57.135714 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:48:57.135714 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:48:57.135714 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:48:57.135714 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:48:57.144923 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 34/69] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:48:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:48:57.144923 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5b29b-2837-4ca5-b21f-1bcac9371473 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:57.144923 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5b29b-2837-4ca5-b21f-1bcac9371473 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:57.144923 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5b29b-2837-4ca5-b21f-1bcac9371473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:57.175271 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-6fc5b29b-2837-4ca5-b21f-1bcac9371473 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:57.177686 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-6fc5b29b-2837-4ca5-b21f-1bcac9371473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:57.216445 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:57.217450 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5035bf3-40c5-417c-b9a8-d860cb4908c0 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:57.260599 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c620d0-cc2e-45e6-9c92-e23c2533ddc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:57.261465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c620d0-cc2e-45e6-9c92-e23c2533ddc1 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:57.261908 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c620d0-cc2e-45e6-9c92-e23c2533ddc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:57.262325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c620d0-cc2e-45e6-9c92-e23c2533ddc1 None admin] PATH_INFO: `/v3/users/9a89bd577d164d69948fbb5e8b571169` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:57.266627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 35/70] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:57.268387 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5b29b-2837-4ca5-b21f-1bcac9371473 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:57.281287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c620d0-cc2e-45e6-9c92-e23c2533ddc1 None admin] RBAC: Authorizing `identity:get_user(user_id=9a89bd577d164d69948fbb5e8b571169)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:57.283034 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c620d0-cc2e-45e6-9c92-e23c2533ddc1 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:57.284929 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 36/71] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:48:57 2022] GET /identity/v3/users/9a89bd577d164d69948fbb5e8b571169 => generated 275 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:57.286321 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d4c33-afce-4afe-bcbf-3d26b49addc3 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:57.286749 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d4c33-afce-4afe-bcbf-3d26b49addc3 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:57.287177 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d4c33-afce-4afe-bcbf-3d26b49addc3 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:57.288527 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 37/72] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:57.290144 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d4c33-afce-4afe-bcbf-3d26b49addc3 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:57.338334 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:57.380597 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5b29b-2837-4ca5-b21f-1bcac9371473 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:57.394736 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d4c33-afce-4afe-bcbf-3d26b49addc3 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:57.414263 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899331f9-a4dd-427f-a89d-fad050958597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:57.414876 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899331f9-a4dd-427f-a89d-fad050958597 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:57.415159 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899331f9-a4dd-427f-a89d-fad050958597 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:57.415435 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899331f9-a4dd-427f-a89d-fad050958597 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:57.416151 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899331f9-a4dd-427f-a89d-fad050958597 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:57.416458 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899331f9-a4dd-427f-a89d-fad050958597 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:57.418019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899331f9-a4dd-427f-a89d-fad050958597 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:57.427482 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 36/73] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:48:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:57.428706 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15ad0b-d151-4a54-a413-ee9b1a976df0 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:57.428989 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15ad0b-d151-4a54-a413-ee9b1a976df0 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:57.429268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15ad0b-d151-4a54-a413-ee9b1a976df0 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:57.432703 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 37/74] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:57.432703 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08578f73-c700-4f0f-8c0c-d803dfcba094 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:57.432703 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08578f73-c700-4f0f-8c0c-d803dfcba094 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:57.432703 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08578f73-c700-4f0f-8c0c-d803dfcba094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:57.434444 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752a106-e17a-49f9-8100-8671a3b7de3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:57.435226 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752a106-e17a-49f9-8100-8671a3b7de3e None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:57.435534 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752a106-e17a-49f9-8100-8671a3b7de3e None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:57.435807 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752a106-e17a-49f9-8100-8671a3b7de3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:57.450886 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-08578f73-c700-4f0f-8c0c-d803dfcba094 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:57.452091 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-08578f73-c700-4f0f-8c0c-d803dfcba094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:57.475174 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:57.511742 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c752a106-e17a-49f9-8100-8671a3b7de3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:57.513400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c752a106-e17a-49f9-8100-8671a3b7de3e None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:57.522709 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 38/75] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:57.526017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08578f73-c700-4f0f-8c0c-d803dfcba094 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:57.565628 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:57.624339 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 38/76] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:48:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:48:57.625252 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752a106-e17a-49f9-8100-8671a3b7de3e None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:57.642499 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08578f73-c700-4f0f-8c0c-d803dfcba094 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:57.683253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc480fa-ba8e-4bc4-8aeb-a4bec6576b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:57.683900 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc480fa-ba8e-4bc4-8aeb-a4bec6576b9e None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:57.684186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc480fa-ba8e-4bc4-8aeb-a4bec6576b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:57.684462 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc480fa-ba8e-4bc4-8aeb-a4bec6576b9e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:57.696894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc480fa-ba8e-4bc4-8aeb-a4bec6576b9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:48:57.703254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-acc480fa-ba8e-4bc4-8aeb-a4bec6576b9e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:48:57.703254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:57.703254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:57.703254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:57.703254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:57.703254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:57.703254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:57.703254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:57.703254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:57.703254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:57.703254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:57.703254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:57.703254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:48:57.703254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:48:57.703254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:48:57.703254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:48:57.704150 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:48:57.704150 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:48:57.704150 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:48:57.704150 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:48:57.704150 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:57.704150 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:57.704150 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:57.704150 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:57.704150 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:57.704150 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:57.704150 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:57.704150 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:57.704150 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:57.704150 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:57.704150 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:57.704150 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:57.704945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:57.704945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:57.704945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:57.704945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:57.704945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:48:57.704945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:48:57.704945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:48:57.704945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:48:57.704945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:48:57.704945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:48:57.704945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:48:57.704945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:48:57.704945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 39/77] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Aug 5 19:48:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:48:57.706322 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168b9fe-168c-4bb4-a891-4a4ef50a15bb None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:57.706608 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168b9fe-168c-4bb4-a891-4a4ef50a15bb None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:57.706884 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168b9fe-168c-4bb4-a891-4a4ef50a15bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:57.727014 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-e168b9fe-168c-4bb4-a891-4a4ef50a15bb None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:57.728238 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-e168b9fe-168c-4bb4-a891-4a4ef50a15bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:57.737880 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752a106-e17a-49f9-8100-8671a3b7de3e None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:57.740811 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:57.774329 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f0d322-ce19-4301-b3e6-f8a70534334e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:57.774883 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f0d322-ce19-4301-b3e6-f8a70534334e None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:57.775170 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f0d322-ce19-4301-b3e6-f8a70534334e None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:57.775454 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f0d322-ce19-4301-b3e6-f8a70534334e None admin] PATH_INFO: `/v3/users/88c3879e2d344041935d28c7f22fdeeb` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:57.776968 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f0d322-ce19-4301-b3e6-f8a70534334e None admin] RBAC: Authorizing `identity:get_user(user_id=88c3879e2d344041935d28c7f22fdeeb)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:57.778468 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f0d322-ce19-4301-b3e6-f8a70534334e None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:57.780118 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 39/78] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:48:57 2022] GET /identity/v3/users/88c3879e2d344041935d28c7f22fdeeb => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:57.780919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f0d322-ce19-4301-b3e6-f8a70534334e None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:57.797726 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:57.808548 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 40/79] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:57.809504 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168b9fe-168c-4bb4-a891-4a4ef50a15bb None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:57.893748 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f0d322-ce19-4301-b3e6-f8a70534334e None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:57.916934 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168b9fe-168c-4bb4-a891-4a4ef50a15bb None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:57.932372 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2b7d1-e931-4183-9dbc-5865e7c6cacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:57.933041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2b7d1-e931-4183-9dbc-5865e7c6cacc None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:57.933326 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2b7d1-e931-4183-9dbc-5865e7c6cacc None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:57.933603 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2b7d1-e931-4183-9dbc-5865e7c6cacc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:57.945928 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe2b7d1-e931-4183-9dbc-5865e7c6cacc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:48:57.951035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-afe2b7d1-e931-4183-9dbc-5865e7c6cacc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:48:57.951035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:57.951035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:57.951035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:57.951035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:57.951035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:57.951035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:57.951035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:57.951035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:57.951035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:57.951035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:57.951035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:57.951035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:48:57.951035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:48:57.951035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:48:57.951035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:48:57.951958 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:48:57.951958 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:48:57.951958 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:48:57.951958 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:48:57.951958 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:57.951958 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:57.951958 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:57.951958 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:57.951958 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:57.951958 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:57.951958 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:57.951958 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:57.951958 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:57.951958 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:57.951958 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:57.951958 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:57.952735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:57.952735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:57.952735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:57.952735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:57.952735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:48:57.952735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:48:57.952735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:48:57.952735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:48:57.952735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:48:57.952735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:48:57.952735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:48:57.952735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:48:57.952735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 40/80] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:48:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:48:57.953362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2b7d1-e931-4183-9dbc-5865e7c6cacc None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:57.958732 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389171f-8d2e-4dc4-bf78-3d8569e777f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:57.959380 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389171f-8d2e-4dc4-bf78-3d8569e777f7 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:57.959660 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389171f-8d2e-4dc4-bf78-3d8569e777f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:57.959934 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389171f-8d2e-4dc4-bf78-3d8569e777f7 None admin] PATH_INFO: `/v3/projects/f542d45b34d849d2aad08bc75ebc275a` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:57.961255 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d389171f-8d2e-4dc4-bf78-3d8569e777f7 None admin] RBAC: Authorizing `identity:get_project(project_id=f542d45b34d849d2aad08bc75ebc275a)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:57.962719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d389171f-8d2e-4dc4-bf78-3d8569e777f7 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:57.964402 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 41/81] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:48:57 2022] GET /identity/v3/projects/f542d45b34d849d2aad08bc75ebc275a => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:57.965207 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389171f-8d2e-4dc4-bf78-3d8569e777f7 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:57.981415 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:58.032826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:58.069298 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389171f-8d2e-4dc4-bf78-3d8569e777f7 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:58.071178 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2b7d1-e931-4183-9dbc-5865e7c6cacc None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:58.103459 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509fad0-152c-4a86-9a87-d3094ae64af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:58.104090 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509fad0-152c-4a86-9a87-d3094ae64af7 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:58.104382 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509fad0-152c-4a86-9a87-d3094ae64af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:58.104689 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509fad0-152c-4a86-9a87-d3094ae64af7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:58.109606 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f94eee-239f-4d1c-a24c-4818f37fc201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:58.110264 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f94eee-239f-4d1c-a24c-4818f37fc201 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:58.111100 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f94eee-239f-4d1c-a24c-4818f37fc201 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:58.111426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f94eee-239f-4d1c-a24c-4818f37fc201 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:58.112231 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f94eee-239f-4d1c-a24c-4818f37fc201 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:58.112606 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f94eee-239f-4d1c-a24c-4818f37fc201 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:58.114040 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f94eee-239f-4d1c-a24c-4818f37fc201 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:58.123048 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 41/82] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:48:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:58.123868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f94eee-239f-4d1c-a24c-4818f37fc201 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:58.177332 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509fad0-152c-4a86-9a87-d3094ae64af7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:58.178819 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509fad0-152c-4a86-9a87-d3094ae64af7 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:58.197502 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:58.237155 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f94eee-239f-4d1c-a24c-4818f37fc201 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:58.272034 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa21b4-7197-401e-8b00-8c0bd9d75c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:58.272719 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa21b4-7197-401e-8b00-8c0bd9d75c7d None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:58.273017 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa21b4-7197-401e-8b00-8c0bd9d75c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:58.273299 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa21b4-7197-401e-8b00-8c0bd9d75c7d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:58.274451 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfa21b4-7197-401e-8b00-8c0bd9d75c7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:58.276188 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfa21b4-7197-401e-8b00-8c0bd9d75c7d None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:58.277825 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 42/83] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:48:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:58.278588 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa21b4-7197-401e-8b00-8c0bd9d75c7d None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:58.285313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:58.290023 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 42/84] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:48:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:48:58.290886 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509fad0-152c-4a86-9a87-d3094ae64af7 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:58.388371 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa21b4-7197-401e-8b00-8c0bd9d75c7d None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:58.401934 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509fad0-152c-4a86-9a87-d3094ae64af7 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:58.424017 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02b461-49f2-421d-a917-3d021175e0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:58.424903 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02b461-49f2-421d-a917-3d021175e0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:58.425192 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02b461-49f2-421d-a917-3d021175e0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:58.425283 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:58.425627 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02b461-49f2-421d-a917-3d021175e0c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:58.426199 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e02b461-49f2-421d-a917-3d021175e0c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:58.426540 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e02b461-49f2-421d-a917-3d021175e0c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:58.427899 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e02b461-49f2-421d-a917-3d021175e0c9 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:58.436604 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 43/85] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Aug 5 19:48:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:58.438207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02b461-49f2-421d-a917-3d021175e0c9 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:58.446645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdfdcd-04ac-4584-ab21-e0f4f9474640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:58.447259 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdfdcd-04ac-4584-ab21-e0f4f9474640 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:58.447536 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdfdcd-04ac-4584-ab21-e0f4f9474640 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:58.447812 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdfdcd-04ac-4584-ab21-e0f4f9474640 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:58.448618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cdfdcd-04ac-4584-ab21-e0f4f9474640 None admin] RBAC: Adding query filter params (role.id=fbd7c81b4fba4b6d9e61b1e9bb3e2046, scope.project.id=f542d45b34d849d2aad08bc75ebc275a, user.id=9a89bd577d164d69948fbb5e8b571169) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:58.448944 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cdfdcd-04ac-4584-ab21-e0f4f9474640 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:58.450373 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cdfdcd-04ac-4584-ab21-e0f4f9474640 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:58.456081 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 43/86] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:48:58 2022] GET /identity/v3/role_assignments?user.id=9a89bd577d164d69948fbb5e8b571169&scope.project.id=f542d45b34d849d2aad08bc75ebc275a&role.id=fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:58.456756 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdfdcd-04ac-4584-ab21-e0f4f9474640 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:58.530919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:58.581057 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdfdcd-04ac-4584-ab21-e0f4f9474640 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:58.583794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02b461-49f2-421d-a917-3d021175e0c9 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:58.622388 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a612f99-71aa-4ac0-a4a0-334d988ef9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:58.622885 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c123d-d8a3-4bec-8f98-76469d1e15b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:58.623502 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a612f99-71aa-4ac0-a4a0-334d988ef9c2 None admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:58.623605 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c123d-d8a3-4bec-8f98-76469d1e15b6 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:58.623824 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c123d-d8a3-4bec-8f98-76469d1e15b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:58.623986 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a612f99-71aa-4ac0-a4a0-334d988ef9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:58.624229 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c123d-d8a3-4bec-8f98-76469d1e15b6 None admin] PATH_INFO: `/v3/users/9a89bd577d164d69948fbb5e8b571169` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:58.624380 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a612f99-71aa-4ac0-a4a0-334d988ef9c2 None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:58.625408 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a612f99-71aa-4ac0-a4a0-334d988ef9c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:58.625585 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7c123d-d8a3-4bec-8f98-76469d1e15b6 None admin] RBAC: Authorizing `identity:get_user(user_id=9a89bd577d164d69948fbb5e8b571169)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:58.626963 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a612f99-71aa-4ac0-a4a0-334d988ef9c2 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:58.627151 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7c123d-d8a3-4bec-8f98-76469d1e15b6 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:58.628999 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 44/87] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:48:58 2022] GET /identity/v3/users/9a89bd577d164d69948fbb5e8b571169 => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:58.629860 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c123d-d8a3-4bec-8f98-76469d1e15b6 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:58.654421 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 44/88] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Aug 5 19:48:58 2022] POST /identity/v3/users => generated 289 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:48:58.655040 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a612f99-71aa-4ac0-a4a0-334d988ef9c2 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:58.715087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:58.761293 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c123d-d8a3-4bec-8f98-76469d1e15b6 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:58.801242 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:58.803198 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d489ad1e-85f3-4014-897e-701987022732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:58.804088 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489ad1e-85f3-4014-897e-701987022732 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:58.804488 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489ad1e-85f3-4014-897e-701987022732 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:58.805476 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489ad1e-85f3-4014-897e-701987022732 None admin] PATH_INFO: `/v3/projects/f542d45b34d849d2aad08bc75ebc275a` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:58.806716 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d489ad1e-85f3-4014-897e-701987022732 None admin] RBAC: Authorizing `identity:get_project(project_id=f542d45b34d849d2aad08bc75ebc275a)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:58.808402 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d489ad1e-85f3-4014-897e-701987022732 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:58.810882 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 45/89] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:48:58 2022] GET /identity/v3/projects/f542d45b34d849d2aad08bc75ebc275a => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:58.812061 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d489ad1e-85f3-4014-897e-701987022732 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:58.821823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a612f99-71aa-4ac0-a4a0-334d988ef9c2 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:58.885143 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0fa122-fa62-4678-b8b6-567ebea827ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:58.886382 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fa122-fa62-4678-b8b6-567ebea827ed None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:58.886914 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fa122-fa62-4678-b8b6-567ebea827ed None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:58.887401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fa122-fa62-4678-b8b6-567ebea827ed None admin] PATH_INFO: `/v3/users/9a89bd577d164d69948fbb5e8b571169` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:58.888808 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0fa122-fa62-4678-b8b6-567ebea827ed None admin] RBAC: Authorizing `identity:get_user(user_id=9a89bd577d164d69948fbb5e8b571169)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:58.890319 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0fa122-fa62-4678-b8b6-567ebea827ed None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:58.893580 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 45/90] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:48:58 2022] GET /identity/v3/users/9a89bd577d164d69948fbb5e8b571169 => generated 275 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:58.894523 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0fa122-fa62-4678-b8b6-567ebea827ed None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:58.945809 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d489ad1e-85f3-4014-897e-701987022732 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:59.009092 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81863a45-967c-4349-9941-a796f9067a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:59.009819 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81863a45-967c-4349-9941-a796f9067a02 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:59.010104 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81863a45-967c-4349-9941-a796f9067a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:59.010538 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81863a45-967c-4349-9941-a796f9067a02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:59.045148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0fa122-fa62-4678-b8b6-567ebea827ed None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:59.049263 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:59.087018 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3060ee3a-ec29-4fde-86a9-f2a84b903da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:59.087678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3060ee3a-ec29-4fde-86a9-f2a84b903da4 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:59.087964 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3060ee3a-ec29-4fde-86a9-f2a84b903da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:59.088248 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3060ee3a-ec29-4fde-86a9-f2a84b903da4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:59.089117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3060ee3a-ec29-4fde-86a9-f2a84b903da4 None admin] RBAC: Adding query filter params (role.id=89919f784ad144529b07c8ab92ee7d48, scope.project.id=f542d45b34d849d2aad08bc75ebc275a, user.id=88c3879e2d344041935d28c7f22fdeeb) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:59.089987 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3060ee3a-ec29-4fde-86a9-f2a84b903da4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:59.090947 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3060ee3a-ec29-4fde-86a9-f2a84b903da4 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:59.097634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 46/91] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:48:58 2022] GET /identity/v3/role_assignments?user.id=88c3879e2d344041935d28c7f22fdeeb&scope.project.id=f542d45b34d849d2aad08bc75ebc275a&role.id=89919f784ad144529b07c8ab92ee7d48 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:59.099344 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3060ee3a-ec29-4fde-86a9-f2a84b903da4 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:59.116716 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:59.122917 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81863a45-967c-4349-9941-a796f9067a02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:59.127144 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81863a45-967c-4349-9941-a796f9067a02 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:59.320120 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3060ee3a-ec29-4fde-86a9-f2a84b903da4 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:59.359341 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 46/92] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:48:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:48:59.365892 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81863a45-967c-4349-9941-a796f9067a02 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:59.410756 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a32b9-4fae-412e-93d4-0c0d9a3cd19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:59.411321 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a32b9-4fae-412e-93d4-0c0d9a3cd19c None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:59.411610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a32b9-4fae-412e-93d4-0c0d9a3cd19c None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:59.411892 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a32b9-4fae-412e-93d4-0c0d9a3cd19c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:59.486340 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:59.536369 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9a32b9-4fae-412e-93d4-0c0d9a3cd19c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:59.538277 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9a32b9-4fae-412e-93d4-0c0d9a3cd19c None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:59.550691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:59.552462 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81863a45-967c-4349-9941-a796f9067a02 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:59.618309 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe1c8b-5892-4a0a-843b-348e726732b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:59.618976 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe1c8b-5892-4a0a-843b-348e726732b0 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:59.619266 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe1c8b-5892-4a0a-843b-348e726732b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:59.619546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe1c8b-5892-4a0a-843b-348e726732b0 None admin] PATH_INFO: `/v3/projects/f542d45b34d849d2aad08bc75ebc275a` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:59.625308 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fe1c8b-5892-4a0a-843b-348e726732b0 None admin] RBAC: Authorizing `identity:get_project(project_id=f542d45b34d849d2aad08bc75ebc275a)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:59.627623 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fe1c8b-5892-4a0a-843b-348e726732b0 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:59.629407 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 47/93] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:48:59 2022] GET /identity/v3/projects/f542d45b34d849d2aad08bc75ebc275a => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:48:59.635461 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe1c8b-5892-4a0a-843b-348e726732b0 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:59.690269 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 47/94] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:48:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:48:59.698574 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a32b9-4fae-412e-93d4-0c0d9a3cd19c None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:59.842645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:59.848654 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe1c8b-5892-4a0a-843b-348e726732b0 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:59.915195 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9973c5-431c-4c3f-b416-7d577251121b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:59.915849 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9973c5-431c-4c3f-b416-7d577251121b None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:59.916140 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9973c5-431c-4c3f-b416-7d577251121b None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:59.916454 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9973c5-431c-4c3f-b416-7d577251121b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:59.917335 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9973c5-431c-4c3f-b416-7d577251121b None admin] RBAC: Adding query filter params (role.id=0eafd8a7ba2d47faaf5aecbb3360143b, scope.project.id=f542d45b34d849d2aad08bc75ebc275a, user.id=9a89bd577d164d69948fbb5e8b571169) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:59.917685 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9973c5-431c-4c3f-b416-7d577251121b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:59.921264 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9973c5-431c-4c3f-b416-7d577251121b None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:59.927351 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a32b9-4fae-412e-93d4-0c0d9a3cd19c None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:59.934343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 48/95] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:48:59 2022] GET /identity/v3/role_assignments?user.id=9a89bd577d164d69948fbb5e8b571169&scope.project.id=f542d45b34d849d2aad08bc75ebc275a&role.id=0eafd8a7ba2d47faaf5aecbb3360143b => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:48:59.993970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0eafb-835c-4734-b715-1f9ac2a7e53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:59.994977 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0eafb-835c-4734-b715-1f9ac2a7e53a None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:59.995457 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0eafb-835c-4734-b715-1f9ac2a7e53a None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:59.995939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0eafb-835c-4734-b715-1f9ac2a7e53a None admin] PATH_INFO: `/v3/projects/ea72a5b1e48a4077bac98d52c5077d43` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:59.997345 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c0eafb-835c-4734-b715-1f9ac2a7e53a None admin] RBAC: Authorizing `identity:get_project(project_id=ea72a5b1e48a4077bac98d52c5077d43)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:59.999234 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c0eafb-835c-4734-b715-1f9ac2a7e53a None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:00.002304 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 48/96] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:48:59 2022] GET /identity/v3/projects/ea72a5b1e48a4077bac98d52c5077d43 => generated 312 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:00.008838 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9973c5-431c-4c3f-b416-7d577251121b None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:00.068840 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:00.151523 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9973c5-431c-4c3f-b416-7d577251121b None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:00.198153 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b618e196-e411-4757-9164-b245d7038574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:00.198787 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618e196-e411-4757-9164-b245d7038574 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:00.199072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618e196-e411-4757-9164-b245d7038574 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:00.199347 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618e196-e411-4757-9164-b245d7038574 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:00.200148 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b618e196-e411-4757-9164-b245d7038574 None admin] RBAC: Adding query filter params (role.id=fbd7c81b4fba4b6d9e61b1e9bb3e2046, scope.project.id=ea72a5b1e48a4077bac98d52c5077d43, user.id=9a89bd577d164d69948fbb5e8b571169) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:00.200463 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b618e196-e411-4757-9164-b245d7038574 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:00.201958 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b618e196-e411-4757-9164-b245d7038574 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:00.208900 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 49/97] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:49:00 2022] GET /identity/v3/role_assignments?user.id=9a89bd577d164d69948fbb5e8b571169&scope.project.id=ea72a5b1e48a4077bac98d52c5077d43&role.id=fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:01.158558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70426561-3648-4645-ab16-8e7717cd6e4d None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:01.159520 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70426561-3648-4645-ab16-8e7717cd6e4d None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:01.159997 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70426561-3648-4645-ab16-8e7717cd6e4d None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:01.161892 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 49/98] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:01.167106 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1ff4b-61f7-4e8c-9e10-40294026437f None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:01.167568 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1ff4b-61f7-4e8c-9e10-40294026437f None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:01.167642 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1ff4b-61f7-4e8c-9e10-40294026437f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:01.210126 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-1fb1ff4b-61f7-4e8c-9e10-40294026437f None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:01.211315 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-1fb1ff4b-61f7-4e8c-9e10-40294026437f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:01.323111 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 50/99] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:49:01.362540 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70426561-3648-4645-ab16-8e7717cd6e4d None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:01.520779 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70426561-3648-4645-ab16-8e7717cd6e4d None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:01.568339 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d159fb-d434-446c-8cb9-43bb6c1115d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:01.569026 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d159fb-d434-446c-8cb9-43bb6c1115d9 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:01.569310 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d159fb-d434-446c-8cb9-43bb6c1115d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:01.569585 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d159fb-d434-446c-8cb9-43bb6c1115d9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:01.572946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:01.595453 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d159fb-d434-446c-8cb9-43bb6c1115d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:01.600569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-85d159fb-d434-446c-8cb9-43bb6c1115d9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:01.600569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:01.600569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:01.600569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:01.600569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:01.600569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:01.600569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:01.600569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:01.600569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:01.600569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:01.600569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:01.600569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:01.600569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:01.600569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:01.600569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:01.600569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:01.601449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:01.601449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:01.601449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:01.601449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:01.601449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:01.601449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:01.601449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:01.601449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:01.601449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:01.601449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:01.601449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:01.601449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:01.601449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:01.601449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:01.601449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:01.601449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:01.602211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:01.602211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:01.602211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:01.602211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:01.602211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:01.602211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:01.602211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:01.602211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:01.602211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:01.602211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:01.602211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:01.602211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:01.602211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 50/100] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:01.609125 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1ff4b-61f7-4e8c-9e10-40294026437f None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:01.647677 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:01.762259 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1ff4b-61f7-4e8c-9e10-40294026437f None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:01.817314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc017ed-3ea2-4529-88cd-079c4b9cfd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:01.817861 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc017ed-3ea2-4529-88cd-079c4b9cfd30 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:01.818141 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc017ed-3ea2-4529-88cd-079c4b9cfd30 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:01.818413 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc017ed-3ea2-4529-88cd-079c4b9cfd30 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:01.819143 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc017ed-3ea2-4529-88cd-079c4b9cfd30 None admin] RBAC: Adding query filter params (name=member) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:01.819516 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc017ed-3ea2-4529-88cd-079c4b9cfd30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:01.820941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc017ed-3ea2-4529-88cd-079c4b9cfd30 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:01.837689 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 51/101] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:01.843270 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d159fb-d434-446c-8cb9-43bb6c1115d9 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:01.977445 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc068db3-403c-4db6-ab12-f0214229dd03 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:01.977933 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc068db3-403c-4db6-ab12-f0214229dd03 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:01.978405 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc068db3-403c-4db6-ab12-f0214229dd03 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:01.981483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 52/102] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:01.989479 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75c67f-b947-4519-b74f-49525dab20d2 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:01.989768 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75c67f-b947-4519-b74f-49525dab20d2 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:01.990055 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75c67f-b947-4519-b74f-49525dab20d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:02.010763 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d159fb-d434-446c-8cb9-43bb6c1115d9 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:02.031508 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-5e75c67f-b947-4519-b74f-49525dab20d2 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:02.033124 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5e75c67f-b947-4519-b74f-49525dab20d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:02.100723 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d49f75d-87e6-44bb-b70b-609783ce8548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:02.101313 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49f75d-87e6-44bb-b70b-609783ce8548 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:02.101594 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49f75d-87e6-44bb-b70b-609783ce8548 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:02.101873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49f75d-87e6-44bb-b70b-609783ce8548 None admin] PATH_INFO: `/v3/users/9a89bd577d164d69948fbb5e8b571169` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:02.109504 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d49f75d-87e6-44bb-b70b-609783ce8548 None admin] RBAC: Authorizing `identity:get_user(user_id=9a89bd577d164d69948fbb5e8b571169)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:02.111547 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d49f75d-87e6-44bb-b70b-609783ce8548 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:02.114666 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 51/103] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:01 2022] GET /identity/v3/users/9a89bd577d164d69948fbb5e8b571169 => generated 275 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:02.118766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d49f75d-87e6-44bb-b70b-609783ce8548 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:02.226032 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 53/104] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:49:02.252620 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:02.281382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75c67f-b947-4519-b74f-49525dab20d2 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:02.320108 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:02.327456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d49f75d-87e6-44bb-b70b-609783ce8548 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:02.380501 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c897dcc-4d15-4916-a7fd-d6660d7c89e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:02.381185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c897dcc-4d15-4916-a7fd-d6660d7c89e7 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:02.381871 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c897dcc-4d15-4916-a7fd-d6660d7c89e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:02.382332 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c897dcc-4d15-4916-a7fd-d6660d7c89e7 None admin] PATH_INFO: `/v3/projects/f542d45b34d849d2aad08bc75ebc275a` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:02.384117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c897dcc-4d15-4916-a7fd-d6660d7c89e7 None admin] RBAC: Authorizing `identity:get_project(project_id=f542d45b34d849d2aad08bc75ebc275a)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:02.385857 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c897dcc-4d15-4916-a7fd-d6660d7c89e7 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:02.388103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 52/105] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:02 2022] GET /identity/v3/projects/f542d45b34d849d2aad08bc75ebc275a => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:02.395445 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c897dcc-4d15-4916-a7fd-d6660d7c89e7 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:02.460567 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75c67f-b947-4519-b74f-49525dab20d2 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:02.498546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ec1d2-25b4-40f4-b387-0ee030fb734b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:02.499480 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ec1d2-25b4-40f4-b387-0ee030fb734b None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:02.499964 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ec1d2-25b4-40f4-b387-0ee030fb734b None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:02.500384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ec1d2-25b4-40f4-b387-0ee030fb734b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:02.502583 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723ec1d2-25b4-40f4-b387-0ee030fb734b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:02.504289 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723ec1d2-25b4-40f4-b387-0ee030fb734b None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:02.506294 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 54/106] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:02.507793 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e09a4d-29b6-4ea0-a233-570dfffecbb0 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:02.508295 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e09a4d-29b6-4ea0-a233-570dfffecbb0 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:02.508763 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e09a4d-29b6-4ea0-a233-570dfffecbb0 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:02.510189 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 55/107] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:02.511456 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc67a5-2c56-45f7-918c-66e58806539c None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:02.511952 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc67a5-2c56-45f7-918c-66e58806539c None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:02.512455 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc67a5-2c56-45f7-918c-66e58806539c None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:02.513848 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 56/108] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:02.515087 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc67a5-2c56-45f7-918c-66e58806539c None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:02.520797 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c897dcc-4d15-4916-a7fd-d6660d7c89e7 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:02.553771 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:02.562361 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f459f86e-9d46-4b02-9bd1-f04e7f73d93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:02.563922 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459f86e-9d46-4b02-9bd1-f04e7f73d93d None admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:02.564671 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459f86e-9d46-4b02-9bd1-f04e7f73d93d None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:02.565169 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459f86e-9d46-4b02-9bd1-f04e7f73d93d None admin] PATH_INFO: `/v3/projects/f542d45b34d849d2aad08bc75ebc275a/users/9a89bd577d164d69948fbb5e8b571169/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:02.567601 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f459f86e-9d46-4b02-9bd1-f04e7f73d93d None admin] RBAC: Authorizing `identity:create_grant(project_id=f542d45b34d849d2aad08bc75ebc275a, user_id=9a89bd577d164d69948fbb5e8b571169, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:02.569394 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f459f86e-9d46-4b02-9bd1-f04e7f73d93d None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:02.586898 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 53/109] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Aug 5 19:49:02 2022] PUT /identity/v3/projects/f542d45b34d849d2aad08bc75ebc275a/users/9a89bd577d164d69948fbb5e8b571169/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:02.588841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f05cf1-7b62-4044-a264-4954d8d85af5 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:02.589369 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f05cf1-7b62-4044-a264-4954d8d85af5 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:02.589968 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f05cf1-7b62-4044-a264-4954d8d85af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:02.611302 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-38f05cf1-7b62-4044-a264-4954d8d85af5 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:02.613023 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-38f05cf1-7b62-4044-a264-4954d8d85af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:02.616682 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:02.638151 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc67a5-2c56-45f7-918c-66e58806539c None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:02.689905 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917df106-f95c-4df4-b950-ce55e8385340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:02.691237 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917df106-f95c-4df4-b950-ce55e8385340 None admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:02.691670 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917df106-f95c-4df4-b950-ce55e8385340 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:02.692098 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917df106-f95c-4df4-b950-ce55e8385340 None admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:02.693286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917df106-f95c-4df4-b950-ce55e8385340 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:02.695822 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917df106-f95c-4df4-b950-ce55e8385340 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:02.714247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 54/110] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:02.716194 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded1898-8328-4b44-ae80-31c76840f9f3 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:02.716691 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded1898-8328-4b44-ae80-31c76840f9f3 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:02.717744 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded1898-8328-4b44-ae80-31c76840f9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:02.739312 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 57/111] 10.222.0.39 () {60 vars in 1141 bytes} [Fri Aug 5 19:49:02 2022] POST /identity/v3/projects => generated 302 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:02.746363 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-3ded1898-8328-4b44-ae80-31c76840f9f3 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:02.748272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-3ded1898-8328-4b44-ae80-31c76840f9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:02.753826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917df106-f95c-4df4-b950-ce55e8385340 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:02.842600 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 55/112] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:02.879015 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded1898-8328-4b44-ae80-31c76840f9f3 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:02.955133 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:02.975406 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917df106-f95c-4df4-b950-ce55e8385340 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:03.026369 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f8ee5d-cc5b-42e7-8060-303345cae2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:03.027655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f8ee5d-cc5b-42e7-8060-303345cae2d3 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:03.027989 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f8ee5d-cc5b-42e7-8060-303345cae2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:03.028270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f8ee5d-cc5b-42e7-8060-303345cae2d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:03.031492 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded1898-8328-4b44-ae80-31c76840f9f3 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:03.037205 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:03.041671 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f8ee5d-cc5b-42e7-8060-303345cae2d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:03.047177 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-f6f8ee5d-cc5b-42e7-8060-303345cae2d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:49:03.047177 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:03.047177 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:03.047177 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:03.047177 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:03.047177 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:03.047177 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:03.047177 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:03.047177 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:03.047177 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:03.047177 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:03.047177 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:03.047177 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:03.047177 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:03.047177 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:03.047177 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:03.048986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:03.048986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:03.048986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:03.048986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:03.048986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:03.048986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:03.048986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:03.048986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:03.048986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:03.048986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:03.048986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:03.048986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:03.048986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:03.048986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:03.048986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:03.048986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:03.049920 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:03.049920 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:03.049920 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:03.049920 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:03.049920 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:03.049920 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:03.049920 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:03.049920 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:03.049920 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:03.049920 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:03.049920 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:49:03.049920 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:49:03.056792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 58/113] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:49:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:03.058895 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f8ee5d-cc5b-42e7-8060-303345cae2d3 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:03.081903 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f53e0-6b8c-47c3-b569-f32fede8e8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:03.082940 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f53e0-6b8c-47c3-b569-f32fede8e8e3 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:03.083499 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f53e0-6b8c-47c3-b569-f32fede8e8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:03.083957 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f53e0-6b8c-47c3-b569-f32fede8e8e3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:03.096933 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889f53e0-6b8c-47c3-b569-f32fede8e8e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:03.103035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-889f53e0-6b8c-47c3-b569-f32fede8e8e3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:49:03.103035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:03.103035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:03.103035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:03.103035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:03.103035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:03.103035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:03.103035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:03.103035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:03.103035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:03.103035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:03.103035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:03.103035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:03.103035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:03.103035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:03.103035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:03.103990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:03.103990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:03.103990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:03.103990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:03.103990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:03.103990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:03.103990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:03.103990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:03.103990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:03.103990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:03.103990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:03.103990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:03.103990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:03.103990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:03.103990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:03.103990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:03.104792 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:03.104792 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:03.104792 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:03.104792 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:03.104792 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:03.104792 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:03.104792 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:03.104792 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:03.104792 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:03.104792 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:03.104792 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:49:03.104792 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:03.107318 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 56/114] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:03.113766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f53e0-6b8c-47c3-b569-f32fede8e8e3 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:03.186190 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f8ee5d-cc5b-42e7-8060-303345cae2d3 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:03.230937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf29e2-2046-44dc-9557-a51e11852684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:03.231552 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf29e2-2046-44dc-9557-a51e11852684 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:03.231797 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf29e2-2046-44dc-9557-a51e11852684 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:03.232025 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf29e2-2046-44dc-9557-a51e11852684 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:03.232781 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf29e2-2046-44dc-9557-a51e11852684 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:03.233117 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf29e2-2046-44dc-9557-a51e11852684 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:03.234559 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf29e2-2046-44dc-9557-a51e11852684 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:03.243801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 59/115] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:03.249549 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf29e2-2046-44dc-9557-a51e11852684 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:03.314462 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f53e0-6b8c-47c3-b569-f32fede8e8e3 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:03.320092 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:03.338469 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:03.372535 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f07d7-1047-4a72-a59b-9ee8a5d1bd6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:03.373691 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f07d7-1047-4a72-a59b-9ee8a5d1bd6e None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:03.374185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f07d7-1047-4a72-a59b-9ee8a5d1bd6e None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:03.374663 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f07d7-1047-4a72-a59b-9ee8a5d1bd6e None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:03.375444 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf29e2-2046-44dc-9557-a51e11852684 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:03.375985 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f07d7-1047-4a72-a59b-9ee8a5d1bd6e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:03.376496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f07d7-1047-4a72-a59b-9ee8a5d1bd6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:03.378336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f07d7-1047-4a72-a59b-9ee8a5d1bd6e None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:03.388600 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 57/116] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:03.390333 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9271c-5bb3-4b1e-ac31-05c2fe911fba None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:03.390923 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9271c-5bb3-4b1e-ac31-05c2fe911fba None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:03.392012 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9271c-5bb3-4b1e-ac31-05c2fe911fba None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:03.393850 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 58/117] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:03.394845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9271c-5bb3-4b1e-ac31-05c2fe911fba None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:03.419023 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7c827-696e-42a8-bfe9-228d97128d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:03.420063 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7c827-696e-42a8-bfe9-228d97128d3d None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:03.420539 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7c827-696e-42a8-bfe9-228d97128d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:03.421049 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7c827-696e-42a8-bfe9-228d97128d3d None admin] PATH_INFO: `/v3/users/88c3879e2d344041935d28c7f22fdeeb` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:03.423645 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b7c827-696e-42a8-bfe9-228d97128d3d None admin] RBAC: Authorizing `identity:get_user(user_id=88c3879e2d344041935d28c7f22fdeeb)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:03.426766 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b7c827-696e-42a8-bfe9-228d97128d3d None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:03.428792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 60/118] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:03 2022] GET /identity/v3/users/88c3879e2d344041935d28c7f22fdeeb => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:03.430134 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd568b2-d5c8-4c60-9275-130167c0b861 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:03.430420 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd568b2-d5c8-4c60-9275-130167c0b861 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:03.430687 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd568b2-d5c8-4c60-9275-130167c0b861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:03.455149 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-8cd568b2-d5c8-4c60-9275-130167c0b861 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:03.456368 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-8cd568b2-d5c8-4c60-9275-130167c0b861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:03.543230 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 61/119] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:03.544040 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd568b2-d5c8-4c60-9275-130167c0b861 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:03.583306 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9271c-5bb3-4b1e-ac31-05c2fe911fba None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:03.650298 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:03.684821 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d056d6b-2fe9-47ca-b9cf-83d5cdc19ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:03.685485 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d056d6b-2fe9-47ca-b9cf-83d5cdc19ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:03.685775 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d056d6b-2fe9-47ca-b9cf-83d5cdc19ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:03.686050 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d056d6b-2fe9-47ca-b9cf-83d5cdc19ef3 None admin] PATH_INFO: `/v3/users/246108d952bf421497f9b844b50802ff` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:03.710964 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d056d6b-2fe9-47ca-b9cf-83d5cdc19ef3 None admin] RBAC: Authorizing `identity:get_user(user_id=246108d952bf421497f9b844b50802ff)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:03.712513 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d056d6b-2fe9-47ca-b9cf-83d5cdc19ef3 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:03.718331 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 59/120] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:03 2022] GET /identity/v3/users/246108d952bf421497f9b844b50802ff => generated 289 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:03.724243 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d056d6b-2fe9-47ca-b9cf-83d5cdc19ef3 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:03.745022 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd568b2-d5c8-4c60-9275-130167c0b861 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:03.775876 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:03.788471 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a95a4-afa2-4000-8163-dc3702381e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:03.789250 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a95a4-afa2-4000-8163-dc3702381e5f None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:03.789531 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a95a4-afa2-4000-8163-dc3702381e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:03.789854 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a95a4-afa2-4000-8163-dc3702381e5f None admin] PATH_INFO: `/v3/projects/f542d45b34d849d2aad08bc75ebc275a` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:03.791150 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69a95a4-afa2-4000-8163-dc3702381e5f None admin] RBAC: Authorizing `identity:get_project(project_id=f542d45b34d849d2aad08bc75ebc275a)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:03.792898 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69a95a4-afa2-4000-8163-dc3702381e5f None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:03.794904 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 62/121] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:03 2022] GET /identity/v3/projects/f542d45b34d849d2aad08bc75ebc275a => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:03.795631 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a95a4-afa2-4000-8163-dc3702381e5f None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:03.871817 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d056d6b-2fe9-47ca-b9cf-83d5cdc19ef3 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:03.922006 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7426953b-ab65-482e-b7ff-c1986e6123f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:03.922956 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426953b-ab65-482e-b7ff-c1986e6123f5 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:03.923240 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426953b-ab65-482e-b7ff-c1986e6123f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:03.923943 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426953b-ab65-482e-b7ff-c1986e6123f5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:03.926475 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a95a4-afa2-4000-8163-dc3702381e5f None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:03.937681 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7426953b-ab65-482e-b7ff-c1986e6123f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:03.945510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-7426953b-ab65-482e-b7ff-c1986e6123f5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:03.945510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:03.945510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:03.945510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:03.945510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:03.945510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:03.945510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:03.945510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:03.945510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:03.945510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:03.945510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:03.945510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:03.945510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:03.945510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:03.945510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:03.945510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:03.946372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:03.946372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:03.946372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:03.946372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:03.946372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:03.946372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:03.946372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:03.946372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:03.946372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:03.946372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:03.946372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:03.946372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:03.946372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:03.946372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:03.946372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:03.946372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:03.947137 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:03.947137 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:03.947137 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:03.947137 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:03.947137 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:03.947137 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:03.947137 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:03.947137 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:03.947137 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:03.947137 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:03.947137 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:03.947137 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:03.948248 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 60/122] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Aug 5 19:49:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:03.948433 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a983798-d4ff-4bd7-9090-bec4246712cf None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:03.952890 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a983798-d4ff-4bd7-9090-bec4246712cf None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:03.953386 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a983798-d4ff-4bd7-9090-bec4246712cf None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:03.955220 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 61/123] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:03.955993 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a983798-d4ff-4bd7-9090-bec4246712cf None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:03.965226 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:03.979394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbc461-ccfc-4adc-89cd-66494cd7b78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:03.980274 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbc461-ccfc-4adc-89cd-66494cd7b78f None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:03.981014 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbc461-ccfc-4adc-89cd-66494cd7b78f None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:03.981302 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbc461-ccfc-4adc-89cd-66494cd7b78f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:04.068701 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:04.083024 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddbc461-ccfc-4adc-89cd-66494cd7b78f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:04.085284 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddbc461-ccfc-4adc-89cd-66494cd7b78f None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:04.088031 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a983798-d4ff-4bd7-9090-bec4246712cf None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:04.147655 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71737ec5-0a87-48fa-8dcf-edc92475b587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:04.148536 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71737ec5-0a87-48fa-8dcf-edc92475b587 None admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:04.149266 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71737ec5-0a87-48fa-8dcf-edc92475b587 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:04.150108 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71737ec5-0a87-48fa-8dcf-edc92475b587 None admin] PATH_INFO: `/v3/projects/f542d45b34d849d2aad08bc75ebc275a/users/88c3879e2d344041935d28c7f22fdeeb/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:04.152527 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71737ec5-0a87-48fa-8dcf-edc92475b587 None admin] RBAC: Authorizing `identity:create_grant(project_id=f542d45b34d849d2aad08bc75ebc275a, user_id=88c3879e2d344041935d28c7f22fdeeb, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:04.155035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71737ec5-0a87-48fa-8dcf-edc92475b587 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:04.174700 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 62/124] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Aug 5 19:49:03 2022] PUT /identity/v3/projects/f542d45b34d849d2aad08bc75ebc275a/users/88c3879e2d344041935d28c7f22fdeeb/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:04.175960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71737ec5-0a87-48fa-8dcf-edc92475b587 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:04.245123 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 63/125] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:04.251151 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0d406-c964-4f51-8590-a9ffd3f45a63 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:04.251151 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0d406-c964-4f51-8590-a9ffd3f45a63 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:04.251151 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0d406-c964-4f51-8590-a9ffd3f45a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:04.293802 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-77a0d406-c964-4f51-8590-a9ffd3f45a63 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:04.295154 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-77a0d406-c964-4f51-8590-a9ffd3f45a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:04.328277 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71737ec5-0a87-48fa-8dcf-edc92475b587 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:04.361709 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:04.376430 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083fb79-38c4-463b-af98-f4597741af98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:04.377670 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083fb79-38c4-463b-af98-f4597741af98 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:04.378167 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083fb79-38c4-463b-af98-f4597741af98 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:04.378622 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083fb79-38c4-463b-af98-f4597741af98 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:04.379541 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0083fb79-38c4-463b-af98-f4597741af98 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:04.380022 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0083fb79-38c4-463b-af98-f4597741af98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:04.381917 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0083fb79-38c4-463b-af98-f4597741af98 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:04.395055 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 63/126] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:04.396648 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083fb79-38c4-463b-af98-f4597741af98 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:04.419617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:04.442757 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 64/127] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:04.443451 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a0d406-c964-4f51-8590-a9ffd3f45a63 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:04.586367 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083fb79-38c4-463b-af98-f4597741af98 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:04.633058 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a0d406-c964-4f51-8590-a9ffd3f45a63 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:04.657989 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0fa5e-8908-43b7-8cac-a1b33f77b416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:04.658660 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0fa5e-8908-43b7-8cac-a1b33f77b416 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:04.658951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0fa5e-8908-43b7-8cac-a1b33f77b416 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:04.659234 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0fa5e-8908-43b7-8cac-a1b33f77b416 None admin] PATH_INFO: `/v3/projects/f542d45b34d849d2aad08bc75ebc275a` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:04.660496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d0fa5e-8908-43b7-8cac-a1b33f77b416 None admin] RBAC: Authorizing `identity:get_project(project_id=f542d45b34d849d2aad08bc75ebc275a)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:04.662132 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d0fa5e-8908-43b7-8cac-a1b33f77b416 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:04.663880 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 64/128] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:04 2022] GET /identity/v3/projects/f542d45b34d849d2aad08bc75ebc275a => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:04.664895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0fa5e-8908-43b7-8cac-a1b33f77b416 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:04.705644 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4c920-ffd0-496b-8ead-d84f0fae3eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:04.706305 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4c920-ffd0-496b-8ead-d84f0fae3eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:04.706655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4c920-ffd0-496b-8ead-d84f0fae3eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:04.706938 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4c920-ffd0-496b-8ead-d84f0fae3eb2 None admin] PATH_INFO: `/v3/users/9a89bd577d164d69948fbb5e8b571169` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:04.708183 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4c920-ffd0-496b-8ead-d84f0fae3eb2 None admin] RBAC: Authorizing `identity:get_user(user_id=9a89bd577d164d69948fbb5e8b571169)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:04.709866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4c920-ffd0-496b-8ead-d84f0fae3eb2 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:04.712763 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 65/129] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:04 2022] GET /identity/v3/users/9a89bd577d164d69948fbb5e8b571169 => generated 275 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:04.713557 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4c920-ffd0-496b-8ead-d84f0fae3eb2 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:04.759241 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:04.845926 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:04.864412 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4c920-ffd0-496b-8ead-d84f0fae3eb2 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:04.885770 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0fa5e-8908-43b7-8cac-a1b33f77b416 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:04.937953 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013d59f-5d61-4c18-b19e-e0ebc24df23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:04.938429 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c04480-3942-49ac-bba8-b7e86e474e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:04.938715 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013d59f-5d61-4c18-b19e-e0ebc24df23d None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:04.939101 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c04480-3942-49ac-bba8-b7e86e474e23 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:04.939217 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013d59f-5d61-4c18-b19e-e0ebc24df23d None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:04.939396 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c04480-3942-49ac-bba8-b7e86e474e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:04.939496 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013d59f-5d61-4c18-b19e-e0ebc24df23d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:04.939680 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c04480-3942-49ac-bba8-b7e86e474e23 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:04.940290 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3013d59f-5d61-4c18-b19e-e0ebc24df23d None admin] RBAC: Adding query filter params (role.id=9de09806c19c49e7aba24d414a65eb5f, scope.project.id=f542d45b34d849d2aad08bc75ebc275a, user.id=246108d952bf421497f9b844b50802ff) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:04.940627 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3013d59f-5d61-4c18-b19e-e0ebc24df23d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:04.942086 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3013d59f-5d61-4c18-b19e-e0ebc24df23d None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:04.951687 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c04480-3942-49ac-bba8-b7e86e474e23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:04.960369 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 66/130] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:49:04 2022] GET /identity/v3/role_assignments?user.id=246108d952bf421497f9b844b50802ff&scope.project.id=f542d45b34d849d2aad08bc75ebc275a&role.id=9de09806c19c49e7aba24d414a65eb5f => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:04.961268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013d59f-5d61-4c18-b19e-e0ebc24df23d None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:04.967098 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-85c04480-3942-49ac-bba8-b7e86e474e23 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:04.967098 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:04.967098 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:04.967098 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:04.967098 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:04.967098 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:04.967098 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:04.967098 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:04.967098 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:04.967098 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:04.967098 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:04.967098 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:04.967098 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:04.967098 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:04.967098 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:04.967098 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:04.968107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:04.968107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:04.968107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:04.968107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:04.968107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:04.968107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:04.968107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:04.968107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:04.968107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:04.968107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:04.968107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:04.968107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:04.968107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:04.968107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:04.968107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:04.968107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:04.968938 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:04.968938 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:04.968938 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:04.968938 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:04.968938 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:04.968938 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:04.968938 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:04.968938 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:04.968938 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:04.968938 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:04.968938 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:04.968938 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:04.970421 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 65/131] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:04.976524 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c04480-3942-49ac-bba8-b7e86e474e23 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:05.182496 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:05.207761 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013d59f-5d61-4c18-b19e-e0ebc24df23d None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:05.225269 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c04480-3942-49ac-bba8-b7e86e474e23 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:05.279529 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5fc8d8-5657-41ca-8ddd-6891db32de8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:05.280939 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5fc8d8-5657-41ca-8ddd-6891db32de8a None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:05.281445 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5fc8d8-5657-41ca-8ddd-6891db32de8a None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:05.281965 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5fc8d8-5657-41ca-8ddd-6891db32de8a None admin] PATH_INFO: `/v3/projects/f542d45b34d849d2aad08bc75ebc275a` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:05.288471 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5fc8d8-5657-41ca-8ddd-6891db32de8a None admin] RBAC: Authorizing `identity:get_project(project_id=f542d45b34d849d2aad08bc75ebc275a)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:05.290498 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5fc8d8-5657-41ca-8ddd-6891db32de8a None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:05.292875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 67/132] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:04 2022] GET /identity/v3/projects/f542d45b34d849d2aad08bc75ebc275a => generated 298 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:05.299801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5fc8d8-5657-41ca-8ddd-6891db32de8a None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:05.308723 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ae22c9-dc6a-4111-9725-50efaef7bb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:05.309484 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae22c9-dc6a-4111-9725-50efaef7bb1a None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:05.309781 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae22c9-dc6a-4111-9725-50efaef7bb1a None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:05.310070 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae22c9-dc6a-4111-9725-50efaef7bb1a None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:05.310962 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ae22c9-dc6a-4111-9725-50efaef7bb1a None admin] RBAC: Adding query filter params (name=member) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:05.311142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ae22c9-dc6a-4111-9725-50efaef7bb1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:05.312533 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ae22c9-dc6a-4111-9725-50efaef7bb1a None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:05.334886 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 66/133] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:05.355092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ae22c9-dc6a-4111-9725-50efaef7bb1a None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:05.358074 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:05.423120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5fc8d8-5657-41ca-8ddd-6891db32de8a None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:05.464296 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b19d3c-1631-47f9-938f-e5c57faac7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:05.465592 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b19d3c-1631-47f9-938f-e5c57faac7e4 None admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:05.465592 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b19d3c-1631-47f9-938f-e5c57faac7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:05.466115 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b19d3c-1631-47f9-938f-e5c57faac7e4 None admin] PATH_INFO: `/v3/projects/f542d45b34d849d2aad08bc75ebc275a/users/9a89bd577d164d69948fbb5e8b571169/roles/0eafd8a7ba2d47faaf5aecbb3360143b` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:05.469007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b19d3c-1631-47f9-938f-e5c57faac7e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f542d45b34d849d2aad08bc75ebc275a, user_id=9a89bd577d164d69948fbb5e8b571169, role_id=0eafd8a7ba2d47faaf5aecbb3360143b)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:05.471131 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b19d3c-1631-47f9-938f-e5c57faac7e4 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:05.495919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 68/134] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Aug 5 19:49:05 2022] PUT /identity/v3/projects/f542d45b34d849d2aad08bc75ebc275a/users/9a89bd577d164d69948fbb5e8b571169/roles/0eafd8a7ba2d47faaf5aecbb3360143b => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:05.497143 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013954d0-adb0-4b11-949b-46a13c69ed81 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:05.497435 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013954d0-adb0-4b11-949b-46a13c69ed81 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:05.497713 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013954d0-adb0-4b11-949b-46a13c69ed81 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:05.499168 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 69/135] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:05.507366 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d070c00-6c76-46de-b806-86ded0bbb8d4 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:05.507680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d070c00-6c76-46de-b806-86ded0bbb8d4 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:05.507922 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d070c00-6c76-46de-b806-86ded0bbb8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:05.554981 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-7d070c00-6c76-46de-b806-86ded0bbb8d4 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:05.556493 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-7d070c00-6c76-46de-b806-86ded0bbb8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:05.583347 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:05.647985 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ae22c9-dc6a-4111-9725-50efaef7bb1a None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:05.701269 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 70/136] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:05.705575 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c60bc-3570-4059-a176-79d2962bb24d None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:05.706301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c60bc-3570-4059-a176-79d2962bb24d None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:05.707873 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c60bc-3570-4059-a176-79d2962bb24d None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:05.709679 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 71/137] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:05.717736 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a91910-cde6-406a-a5d5-642043155c73 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:05.720904 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a91910-cde6-406a-a5d5-642043155c73 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:05.721482 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a91910-cde6-406a-a5d5-642043155c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:05.734937 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198852f9-cffc-48c6-afdf-9722ba757d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:05.737351 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198852f9-cffc-48c6-afdf-9722ba757d1b None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:05.741018 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198852f9-cffc-48c6-afdf-9722ba757d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:05.741524 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198852f9-cffc-48c6-afdf-9722ba757d1b None admin] PATH_INFO: `/v3/users/9a89bd577d164d69948fbb5e8b571169` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:05.743095 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198852f9-cffc-48c6-afdf-9722ba757d1b None admin] RBAC: Authorizing `identity:get_user(user_id=9a89bd577d164d69948fbb5e8b571169)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:05.744818 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198852f9-cffc-48c6-afdf-9722ba757d1b None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:05.748147 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 67/138] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:05 2022] GET /identity/v3/users/9a89bd577d164d69948fbb5e8b571169 => generated 275 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:05.749151 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198852f9-cffc-48c6-afdf-9722ba757d1b None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:05.757052 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-13a91910-cde6-406a-a5d5-642043155c73 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:05.758572 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-13a91910-cde6-406a-a5d5-642043155c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:05.831037 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:05.857402 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 72/139] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:05.859486 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a91910-cde6-406a-a5d5-642043155c73 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:05.869758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198852f9-cffc-48c6-afdf-9722ba757d1b None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:05.925735 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb1a67-c109-4df3-b980-855800be62c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:05.926400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb1a67-c109-4df3-b980-855800be62c6 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:05.926690 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb1a67-c109-4df3-b980-855800be62c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:05.926971 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb1a67-c109-4df3-b980-855800be62c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:05.928138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb1a67-c109-4df3-b980-855800be62c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:05.929680 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb1a67-c109-4df3-b980-855800be62c6 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:05.931332 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 68/140] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:05.932087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb1a67-c109-4df3-b980-855800be62c6 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:06.010956 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:06.045290 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb1a67-c109-4df3-b980-855800be62c6 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:06.054937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a91910-cde6-406a-a5d5-642043155c73 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:06.081849 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9e707-80ae-4ec5-8437-4e1e180aa6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:06.082499 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9e707-80ae-4ec5-8437-4e1e180aa6a6 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:06.082796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9e707-80ae-4ec5-8437-4e1e180aa6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:06.083063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9e707-80ae-4ec5-8437-4e1e180aa6a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:06.090512 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:06.094196 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9e707-80ae-4ec5-8437-4e1e180aa6a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:06.100468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-c2b9e707-80ae-4ec5-8437-4e1e180aa6a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:06.100468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:06.100468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:06.100468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:06.100468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:06.100468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:06.100468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:06.100468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:06.100468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:06.100468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:06.100468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:06.100468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:06.100468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:06.100468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:06.100468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:06.100468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:06.101521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:06.101521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:06.101521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:06.101521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:06.101521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:06.101521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:06.101521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:06.101521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:06.101521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:06.101521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:06.101521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:06.101521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:06.101521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:06.101521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:06.101521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:06.101521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:06.102427 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:06.102427 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:06.102427 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:06.102427 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:06.102427 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:06.102427 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:06.102427 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:06.102427 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:06.102427 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:06.102427 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:06.102427 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:06.102427 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:06.108621 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 69/141] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:06.108621 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9e707-80ae-4ec5-8437-4e1e180aa6a6 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:06.146596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b3b61-e2fb-4e60-8933-0f5ad080cc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:06.147261 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b3b61-e2fb-4e60-8933-0f5ad080cc73 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:06.147545 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b3b61-e2fb-4e60-8933-0f5ad080cc73 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:06.147823 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b3b61-e2fb-4e60-8933-0f5ad080cc73 None admin] PATH_INFO: `/v3/projects/ea72a5b1e48a4077bac98d52c5077d43` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:06.151378 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5b3b61-e2fb-4e60-8933-0f5ad080cc73 None admin] RBAC: Authorizing `identity:get_project(project_id=ea72a5b1e48a4077bac98d52c5077d43)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:06.153401 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5b3b61-e2fb-4e60-8933-0f5ad080cc73 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:06.158239 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 73/142] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:05 2022] GET /identity/v3/projects/ea72a5b1e48a4077bac98d52c5077d43 => generated 312 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:06.159078 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b3b61-e2fb-4e60-8933-0f5ad080cc73 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:06.242324 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9e707-80ae-4ec5-8437-4e1e180aa6a6 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:06.281018 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89d9fd-fcbe-4541-b7d9-15fc3cea5470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:06.281727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89d9fd-fcbe-4541-b7d9-15fc3cea5470 None admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:06.282198 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89d9fd-fcbe-4541-b7d9-15fc3cea5470 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:06.282282 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89d9fd-fcbe-4541-b7d9-15fc3cea5470 None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:06.283079 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f89d9fd-fcbe-4541-b7d9-15fc3cea5470 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:06.284556 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f89d9fd-fcbe-4541-b7d9-15fc3cea5470 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:06.310453 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 70/143] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Aug 5 19:49:06 2022] POST /identity/v3/users => generated 283 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:06.311450 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89d9fd-fcbe-4541-b7d9-15fc3cea5470 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:06.420338 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:06.456913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b3b61-e2fb-4e60-8933-0f5ad080cc73 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:06.531143 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89d9fd-fcbe-4541-b7d9-15fc3cea5470 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:06.582801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76fa70-a8ee-4b6f-8a00-d613de8d80d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:06.583709 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76fa70-a8ee-4b6f-8a00-d613de8d80d0 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:06.583997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76fa70-a8ee-4b6f-8a00-d613de8d80d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:06.584280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76fa70-a8ee-4b6f-8a00-d613de8d80d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:06.585220 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a76fa70-a8ee-4b6f-8a00-d613de8d80d0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:06.585566 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a76fa70-a8ee-4b6f-8a00-d613de8d80d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:06.586987 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a76fa70-a8ee-4b6f-8a00-d613de8d80d0 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:06.599637 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 74/144] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 441 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:06.609979 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76fa70-a8ee-4b6f-8a00-d613de8d80d0 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:06.623172 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:06.625777 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4ab9d4-4c3d-4981-bacc-f7874ec875da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:06.626511 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ab9d4-4c3d-4981-bacc-f7874ec875da None admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:06.626800 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ab9d4-4c3d-4981-bacc-f7874ec875da None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:06.627079 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ab9d4-4c3d-4981-bacc-f7874ec875da None admin] PATH_INFO: `/v3/projects/ea72a5b1e48a4077bac98d52c5077d43/users/9a89bd577d164d69948fbb5e8b571169/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:06.629191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4ab9d4-4c3d-4981-bacc-f7874ec875da None admin] RBAC: Authorizing `identity:create_grant(project_id=ea72a5b1e48a4077bac98d52c5077d43, user_id=9a89bd577d164d69948fbb5e8b571169, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:06.630831 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4ab9d4-4c3d-4981-bacc-f7874ec875da None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:06.664236 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 71/145] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Aug 5 19:49:06 2022] PUT /identity/v3/projects/ea72a5b1e48a4077bac98d52c5077d43/users/9a89bd577d164d69948fbb5e8b571169/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:06.689666 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2d37f-c755-44af-9330-9770b6b7550d None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:06.689905 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2d37f-c755-44af-9330-9770b6b7550d None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:06.690193 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2d37f-c755-44af-9330-9770b6b7550d None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:06.692071 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 72/146] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:06.705409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019314c-ba93-4f50-b361-18ac80eeec74 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:06.705747 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019314c-ba93-4f50-b361-18ac80eeec74 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:06.705970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019314c-ba93-4f50-b361-18ac80eeec74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:06.736470 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-3019314c-ba93-4f50-b361-18ac80eeec74 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:06.737893 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-3019314c-ba93-4f50-b361-18ac80eeec74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:06.777974 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76fa70-a8ee-4b6f-8a00-d613de8d80d0 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:06.868373 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0275abfd-b705-4b63-a131-32d190fc4363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:06.869236 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275abfd-b705-4b63-a131-32d190fc4363 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:06.869524 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275abfd-b705-4b63-a131-32d190fc4363 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:06.869808 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275abfd-b705-4b63-a131-32d190fc4363 None admin] PATH_INFO: `/v3/users/9a89bd577d164d69948fbb5e8b571169` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:06.872681 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0275abfd-b705-4b63-a131-32d190fc4363 None admin] RBAC: Authorizing `identity:get_user(user_id=9a89bd577d164d69948fbb5e8b571169)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:06.874198 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0275abfd-b705-4b63-a131-32d190fc4363 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:06.880763 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 75/147] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:06 2022] GET /identity/v3/users/9a89bd577d164d69948fbb5e8b571169 => generated 275 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:06.886039 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0275abfd-b705-4b63-a131-32d190fc4363 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:06.897991 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 73/148] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:49:06.928152 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019314c-ba93-4f50-b361-18ac80eeec74 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:06.994491 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:07.033739 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:07.037145 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0275abfd-b705-4b63-a131-32d190fc4363 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:07.091243 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcacfcc-6014-44d9-8925-d418561e31a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:07.091870 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcacfcc-6014-44d9-8925-d418561e31a7 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:07.092149 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcacfcc-6014-44d9-8925-d418561e31a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:07.092423 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcacfcc-6014-44d9-8925-d418561e31a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:07.153852 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019314c-ba93-4f50-b361-18ac80eeec74 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:07.187661 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcacfcc-6014-44d9-8925-d418561e31a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:07.189232 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcacfcc-6014-44d9-8925-d418561e31a7 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:07.226728 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226d2e6-3d1f-4ad3-ab10-d3663ee06181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:07.227382 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226d2e6-3d1f-4ad3-ab10-d3663ee06181 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:07.227662 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226d2e6-3d1f-4ad3-ab10-d3663ee06181 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:07.227939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226d2e6-3d1f-4ad3-ab10-d3663ee06181 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:07.250384 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3226d2e6-3d1f-4ad3-ab10-d3663ee06181 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:07.258409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-3226d2e6-3d1f-4ad3-ab10-d3663ee06181 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:49:07.258409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:07.258409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:07.258409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:07.258409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:07.258409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:07.258409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:07.258409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:07.258409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:07.258409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:07.258409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:07.258409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:07.258409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:07.258409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:07.258409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:07.258409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:07.259291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:07.259291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:07.259291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:07.259291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:07.259291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:07.259291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:07.259291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:07.259291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:07.259291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:07.259291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:07.259291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:07.259291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:07.259291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:07.259291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:07.259291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:07.259291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:07.260042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:07.260042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:07.260042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:07.260042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:07.260042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:07.260042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:07.260042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:07.260042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:07.260042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:07.260042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:07.260042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:49:07.260042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:07.260042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 74/149] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:49:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:07.270000 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226d2e6-3d1f-4ad3-ab10-d3663ee06181 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:07.334815 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:07.411590 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 76/150] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:07.443337 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcacfcc-6014-44d9-8925-d418561e31a7 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:07.481539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226d2e6-3d1f-4ad3-ab10-d3663ee06181 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:07.490692 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:07.560525 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857d009-2ee4-4cb7-8623-6a92f62e52d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:07.561276 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857d009-2ee4-4cb7-8623-6a92f62e52d9 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:07.561556 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857d009-2ee4-4cb7-8623-6a92f62e52d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:07.561834 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857d009-2ee4-4cb7-8623-6a92f62e52d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:07.562553 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1857d009-2ee4-4cb7-8623-6a92f62e52d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:07.562858 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1857d009-2ee4-4cb7-8623-6a92f62e52d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:07.564420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1857d009-2ee4-4cb7-8623-6a92f62e52d9 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:07.572382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcacfcc-6014-44d9-8925-d418561e31a7 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:07.583191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 75/151] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:07.589451 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857d009-2ee4-4cb7-8623-6a92f62e52d9 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:07.638775 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ee53e-1c57-4580-b20c-56201b5ee318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:07.639659 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ee53e-1c57-4580-b20c-56201b5ee318 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:07.640178 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ee53e-1c57-4580-b20c-56201b5ee318 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:07.640621 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ee53e-1c57-4580-b20c-56201b5ee318 None admin] PATH_INFO: `/v3/projects/f542d45b34d849d2aad08bc75ebc275a` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:07.644521 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ee53e-1c57-4580-b20c-56201b5ee318 None admin] RBAC: Authorizing `identity:get_project(project_id=f542d45b34d849d2aad08bc75ebc275a)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:07.647386 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ee53e-1c57-4580-b20c-56201b5ee318 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:07.653142 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 77/152] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:07 2022] GET /identity/v3/projects/f542d45b34d849d2aad08bc75ebc275a => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:07.680690 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ee53e-1c57-4580-b20c-56201b5ee318 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:07.772941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:07.804400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857d009-2ee4-4cb7-8623-6a92f62e52d9 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:07.855451 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c8436-1901-4f04-a73d-00efd8bfadbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:07.856115 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c8436-1901-4f04-a73d-00efd8bfadbd None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:07.856401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c8436-1901-4f04-a73d-00efd8bfadbd None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:07.856688 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c8436-1901-4f04-a73d-00efd8bfadbd None admin] PATH_INFO: `/v3/users/88c3879e2d344041935d28c7f22fdeeb` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:07.857927 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898c8436-1901-4f04-a73d-00efd8bfadbd None admin] RBAC: Authorizing `identity:get_user(user_id=88c3879e2d344041935d28c7f22fdeeb)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:07.859395 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898c8436-1901-4f04-a73d-00efd8bfadbd None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:07.861334 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 76/153] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:07 2022] GET /identity/v3/users/88c3879e2d344041935d28c7f22fdeeb => generated 247 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:07.867336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c8436-1901-4f04-a73d-00efd8bfadbd None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:07.913692 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:07.936297 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ee53e-1c57-4580-b20c-56201b5ee318 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:07.991880 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c8436-1901-4f04-a73d-00efd8bfadbd None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:08.033068 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab0ec5-d652-404c-bee5-64e2a36507dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:08.033706 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab0ec5-d652-404c-bee5-64e2a36507dd None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:08.033980 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab0ec5-d652-404c-bee5-64e2a36507dd None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:08.034258 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab0ec5-d652-404c-bee5-64e2a36507dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:08.056336 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13108ef9-e095-48fa-a723-7de6164aef24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:08.057830 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13108ef9-e095-48fa-a723-7de6164aef24 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:08.058529 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13108ef9-e095-48fa-a723-7de6164aef24 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:08.059250 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13108ef9-e095-48fa-a723-7de6164aef24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:08.060997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13108ef9-e095-48fa-a723-7de6164aef24 None admin] RBAC: Adding query filter params (role.id=fbd7c81b4fba4b6d9e61b1e9bb3e2046, scope.project.id=f542d45b34d849d2aad08bc75ebc275a, user.id=9a89bd577d164d69948fbb5e8b571169) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:08.061822 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13108ef9-e095-48fa-a723-7de6164aef24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:08.073932 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13108ef9-e095-48fa-a723-7de6164aef24 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:08.088537 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 78/154] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:49:07 2022] GET /identity/v3/role_assignments?user.id=9a89bd577d164d69948fbb5e8b571169&scope.project.id=f542d45b34d849d2aad08bc75ebc275a&role.id=fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 608 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:08.114272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab0ec5-d652-404c-bee5-64e2a36507dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:08.115834 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab0ec5-d652-404c-bee5-64e2a36507dd None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:08.154780 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:08.233786 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 77/155] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:08.241430 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13108ef9-e095-48fa-a723-7de6164aef24 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:08.374337 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c55c5-3df0-4ec6-87b0-8eeb7f3e84e7 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:08.374609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c55c5-3df0-4ec6-87b0-8eeb7f3e84e7 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:08.374984 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c55c5-3df0-4ec6-87b0-8eeb7f3e84e7 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:08.377099 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 78/156] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:49:08.386314 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768263e4-bc2e-4658-bbcd-7092aae006ab None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:08.386581 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768263e4-bc2e-4658-bbcd-7092aae006ab None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:08.386938 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768263e4-bc2e-4658-bbcd-7092aae006ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:08.423567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-768263e4-bc2e-4658-bbcd-7092aae006ab None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:08.424688 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-768263e4-bc2e-4658-bbcd-7092aae006ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:08.535254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 79/157] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 05 19:49:08.543990 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13108ef9-e095-48fa-a723-7de6164aef24 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:08.563000 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:08.568606 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768263e4-bc2e-4658-bbcd-7092aae006ab None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:08.664003 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8920499-b443-4d4f-86fd-5f06297daa7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:08.665419 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8920499-b443-4d4f-86fd-5f06297daa7e None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:08.665730 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:08.666114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8920499-b443-4d4f-86fd-5f06297daa7e None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:08.666596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8920499-b443-4d4f-86fd-5f06297daa7e None admin] PATH_INFO: `/v3/projects/f542d45b34d849d2aad08bc75ebc275a` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:08.668010 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8920499-b443-4d4f-86fd-5f06297daa7e None admin] RBAC: Authorizing `identity:get_project(project_id=f542d45b34d849d2aad08bc75ebc275a)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:08.669676 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8920499-b443-4d4f-86fd-5f06297daa7e None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:08.671851 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 79/158] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:08 2022] GET /identity/v3/projects/f542d45b34d849d2aad08bc75ebc275a => generated 298 bytes in 431 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:08.681635 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8920499-b443-4d4f-86fd-5f06297daa7e None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:08.722967 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768263e4-bc2e-4658-bbcd-7092aae006ab None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:08.760376 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f21b1-ba4a-4d6f-b50f-cb30279d9372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:08.761134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f21b1-ba4a-4d6f-b50f-cb30279d9372 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:08.761424 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f21b1-ba4a-4d6f-b50f-cb30279d9372 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:08.761702 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f21b1-ba4a-4d6f-b50f-cb30279d9372 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:08.774786 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612f21b1-ba4a-4d6f-b50f-cb30279d9372 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:08.780617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-612f21b1-ba4a-4d6f-b50f-cb30279d9372 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:49:08.780617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:08.780617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:08.780617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:08.780617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:08.780617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:08.780617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:08.780617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:08.780617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:08.780617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:08.780617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:08.780617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:08.780617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:08.780617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:08.780617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:08.780617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:08.781481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:08.781481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:08.781481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:08.781481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:08.781481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:08.781481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:08.781481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:08.781481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:08.781481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:08.781481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:08.781481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:08.781481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:08.781481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:08.781481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:08.781481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:08.781481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:08.782318 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:08.782318 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:08.782318 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:08.782318 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:08.782318 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:08.782318 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:08.782318 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:08.782318 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:08.782318 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:08.782318 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:08.782318 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:49:08.782318 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:08.782318 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 80/159] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:08.788536 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f21b1-ba4a-4d6f-b50f-cb30279d9372 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:08.910441 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f21b1-ba4a-4d6f-b50f-cb30279d9372 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:08.945053 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:08.957081 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4484d8f-1bc9-47ef-93ce-6a4ed60ee872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:08.957850 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4484d8f-1bc9-47ef-93ce-6a4ed60ee872 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:08.958226 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4484d8f-1bc9-47ef-93ce-6a4ed60ee872 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:08.958623 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4484d8f-1bc9-47ef-93ce-6a4ed60ee872 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:08.959736 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4484d8f-1bc9-47ef-93ce-6a4ed60ee872 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:08.960114 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4484d8f-1bc9-47ef-93ce-6a4ed60ee872 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:08.962137 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4484d8f-1bc9-47ef-93ce-6a4ed60ee872 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:08.980627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 81/160] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:08.986864 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4484d8f-1bc9-47ef-93ce-6a4ed60ee872 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:08.989608 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8920499-b443-4d4f-86fd-5f06297daa7e None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:09.046735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87059103-d635-4736-a668-c6e710f01798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:09.048200 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87059103-d635-4736-a668-c6e710f01798 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:09.048692 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87059103-d635-4736-a668-c6e710f01798 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:09.049135 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87059103-d635-4736-a668-c6e710f01798 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:09.050096 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87059103-d635-4736-a668-c6e710f01798 None admin] RBAC: Adding query filter params (role.id=89919f784ad144529b07c8ab92ee7d48, scope.project.id=f542d45b34d849d2aad08bc75ebc275a, user.id=88c3879e2d344041935d28c7f22fdeeb) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:09.050575 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87059103-d635-4736-a668-c6e710f01798 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:09.052192 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87059103-d635-4736-a668-c6e710f01798 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:09.062988 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 80/161] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:49:08 2022] GET /identity/v3/role_assignments?user.id=88c3879e2d344041935d28c7f22fdeeb&scope.project.id=f542d45b34d849d2aad08bc75ebc275a&role.id=89919f784ad144529b07c8ab92ee7d48 => generated 608 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:09.178453 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4484d8f-1bc9-47ef-93ce-6a4ed60ee872 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:09.225776 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ff1d4-bab1-482a-a021-55c0b2919a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:09.226444 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ff1d4-bab1-482a-a021-55c0b2919a2c None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:09.226728 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ff1d4-bab1-482a-a021-55c0b2919a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:09.227007 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ff1d4-bab1-482a-a021-55c0b2919a2c None admin] PATH_INFO: `/v3/users/246108d952bf421497f9b844b50802ff` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:09.233132 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ff1d4-bab1-482a-a021-55c0b2919a2c None admin] RBAC: Authorizing `identity:get_user(user_id=246108d952bf421497f9b844b50802ff)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:09.234795 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ff1d4-bab1-482a-a021-55c0b2919a2c None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:09.236985 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 82/162] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:08 2022] GET /identity/v3/users/246108d952bf421497f9b844b50802ff => generated 289 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:09.242364 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87059103-d635-4736-a668-c6e710f01798 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:09.310283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:09.381695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87059103-d635-4736-a668-c6e710f01798 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:09.417950 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc644d-b341-49ac-a5ee-fbd4ff0853a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:09.418859 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc644d-b341-49ac-a5ee-fbd4ff0853a5 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:09.419282 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc644d-b341-49ac-a5ee-fbd4ff0853a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:09.419698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc644d-b341-49ac-a5ee-fbd4ff0853a5 None admin] PATH_INFO: `/v3/projects/f542d45b34d849d2aad08bc75ebc275a` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:09.421017 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc644d-b341-49ac-a5ee-fbd4ff0853a5 None admin] RBAC: Authorizing `identity:get_project(project_id=f542d45b34d849d2aad08bc75ebc275a)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:09.423184 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc644d-b341-49ac-a5ee-fbd4ff0853a5 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:09.425721 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 81/163] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:09 2022] GET /identity/v3/projects/f542d45b34d849d2aad08bc75ebc275a => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:09.431364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ff1d4-bab1-482a-a021-55c0b2919a2c None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:09.585080 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:09.642020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ff1d4-bab1-482a-a021-55c0b2919a2c None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:09.685921 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac8622-8d7d-436d-af7c-c94d82d819a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:09.686558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac8622-8d7d-436d-af7c-c94d82d819a6 None admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:09.686844 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac8622-8d7d-436d-af7c-c94d82d819a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:09.687120 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac8622-8d7d-436d-af7c-c94d82d819a6 None admin] PATH_INFO: `/v3/projects/f542d45b34d849d2aad08bc75ebc275a/users/246108d952bf421497f9b844b50802ff/roles/9de09806c19c49e7aba24d414a65eb5f` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:09.693870 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac8622-8d7d-436d-af7c-c94d82d819a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=f542d45b34d849d2aad08bc75ebc275a, user_id=246108d952bf421497f9b844b50802ff, role_id=9de09806c19c49e7aba24d414a65eb5f)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:09.695330 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac8622-8d7d-436d-af7c-c94d82d819a6 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:09.717621 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 83/164] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Aug 5 19:49:09 2022] PUT /identity/v3/projects/f542d45b34d849d2aad08bc75ebc275a/users/246108d952bf421497f9b844b50802ff/roles/9de09806c19c49e7aba24d414a65eb5f => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:09.729124 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c225e7-6da2-47c5-9564-abf2651d5625 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:09.729629 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c225e7-6da2-47c5-9564-abf2651d5625 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:09.729741 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c225e7-6da2-47c5-9564-abf2651d5625 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:09.731346 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 82/165] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:09.737448 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dabd7-ce8d-440d-b1dd-69dacbef90d5 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:09.737817 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dabd7-ce8d-440d-b1dd-69dacbef90d5 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:09.738110 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dabd7-ce8d-440d-b1dd-69dacbef90d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:09.762444 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-ec3dabd7-ce8d-440d-b1dd-69dacbef90d5 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:09.763654 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-ec3dabd7-ce8d-440d-b1dd-69dacbef90d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:09.843116 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 84/166] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:09.882111 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c225e7-6da2-47c5-9564-abf2651d5625 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:10.015048 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c225e7-6da2-47c5-9564-abf2651d5625 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:10.022836 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:10.068794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc24dc6-4cac-4c79-99c0-2cc9ec8483a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:10.069474 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc24dc6-4cac-4c79-99c0-2cc9ec8483a8 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:10.069755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc24dc6-4cac-4c79-99c0-2cc9ec8483a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:10.070031 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc24dc6-4cac-4c79-99c0-2cc9ec8483a8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:10.081460 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc24dc6-4cac-4c79-99c0-2cc9ec8483a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:10.087007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-bcc24dc6-4cac-4c79-99c0-2cc9ec8483a8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:10.087007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:10.087007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:10.087007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:10.087007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:10.087007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:10.087007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:10.087007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:10.087007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:10.087007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:10.087007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:10.087007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:10.087007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:10.087007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:10.087007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:10.087007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:10.087866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:10.087866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:10.087866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:10.087866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:10.087866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:10.087866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:10.087866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:10.087866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:10.087866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:10.087866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:10.087866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:10.087866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:10.087866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:10.087866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:10.087866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:10.087866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:10.089351 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:10.089351 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:10.089351 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:10.089351 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:10.089351 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:10.089351 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:10.089351 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:10.089351 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:10.089351 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:10.089351 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:10.089351 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:10.089351 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:49:10.089351 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 83/167] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Aug 5 19:49:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:10.096351 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3dabd7-ce8d-440d-b1dd-69dacbef90d5 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:10.192442 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:10.210840 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7656fa4-87ab-4d48-a000-3288453801bb None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:10.211147 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7656fa4-87ab-4d48-a000-3288453801bb None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:10.211382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7656fa4-87ab-4d48-a000-3288453801bb None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:10.213616 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 84/168] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:49:10.218566 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f401c47-137d-4e69-81e5-61945174617a None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:10.218848 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f401c47-137d-4e69-81e5-61945174617a None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:10.219170 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f401c47-137d-4e69-81e5-61945174617a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:10.242304 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-4f401c47-137d-4e69-81e5-61945174617a None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:10.243492 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-4f401c47-137d-4e69-81e5-61945174617a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:10.277010 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3dabd7-ce8d-440d-b1dd-69dacbef90d5 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:10.318617 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 85/169] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:49:10.341453 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f401c47-137d-4e69-81e5-61945174617a None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:10.351900 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddad0bc1-2bc0-4771-9600-8e9c9ab4c437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:10.352459 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad0bc1-2bc0-4771-9600-8e9c9ab4c437 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:10.352758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad0bc1-2bc0-4771-9600-8e9c9ab4c437 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:10.353028 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad0bc1-2bc0-4771-9600-8e9c9ab4c437 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:10.353750 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddad0bc1-2bc0-4771-9600-8e9c9ab4c437 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:10.354048 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddad0bc1-2bc0-4771-9600-8e9c9ab4c437 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:10.355413 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddad0bc1-2bc0-4771-9600-8e9c9ab4c437 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:10.372396 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 85/170] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:10.378284 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddad0bc1-2bc0-4771-9600-8e9c9ab4c437 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:10.492982 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:10.498692 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f401c47-137d-4e69-81e5-61945174617a None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:10.535141 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15349dd9-837b-4b90-ab3c-404a53ca2f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:10.535984 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15349dd9-837b-4b90-ab3c-404a53ca2f2b None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:10.536360 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15349dd9-837b-4b90-ab3c-404a53ca2f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:10.536755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15349dd9-837b-4b90-ab3c-404a53ca2f2b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:10.547367 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15349dd9-837b-4b90-ab3c-404a53ca2f2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:10.552696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-15349dd9-837b-4b90-ab3c-404a53ca2f2b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:10.552696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:10.552696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:10.552696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:10.552696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:10.552696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:10.552696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:10.552696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:10.552696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:10.552696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:10.552696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:10.552696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:10.552696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:10.552696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:10.552696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:10.552696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:10.553610 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:10.553610 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:10.553610 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:10.553610 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:10.553610 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:10.553610 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:10.553610 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:10.553610 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:10.553610 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:10.553610 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:10.553610 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:10.553610 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:10.553610 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:10.553610 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:10.553610 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:10.553610 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:10.554705 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:10.554705 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:10.554705 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:10.554705 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:10.554705 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:10.554705 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:10.554705 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:10.554705 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:10.554705 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:10.554705 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:10.554705 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:10.554705 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:49:10.554705 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 86/171] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:10.560083 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15349dd9-837b-4b90-ab3c-404a53ca2f2b None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:10.577829 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddad0bc1-2bc0-4771-9600-8e9c9ab4c437 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:10.587204 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:10.643724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2676cbc-126b-4381-8587-b5f402c98b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:10.644551 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2676cbc-126b-4381-8587-b5f402c98b28 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:10.644862 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2676cbc-126b-4381-8587-b5f402c98b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:10.645139 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2676cbc-126b-4381-8587-b5f402c98b28 None admin] PATH_INFO: `/v3/users/9a89bd577d164d69948fbb5e8b571169` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:10.646565 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2676cbc-126b-4381-8587-b5f402c98b28 None admin] RBAC: Authorizing `identity:get_user(user_id=9a89bd577d164d69948fbb5e8b571169)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:10.648034 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2676cbc-126b-4381-8587-b5f402c98b28 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:10.650020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 86/172] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:10 2022] GET /identity/v3/users/9a89bd577d164d69948fbb5e8b571169 => generated 275 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:10.656089 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2676cbc-126b-4381-8587-b5f402c98b28 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:10.671460 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15349dd9-837b-4b90-ab3c-404a53ca2f2b None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:10.708319 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360bb098-1614-4894-ae5c-dbc6997f46f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:10.709121 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360bb098-1614-4894-ae5c-dbc6997f46f4 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:10.709458 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360bb098-1614-4894-ae5c-dbc6997f46f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:10.709911 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360bb098-1614-4894-ae5c-dbc6997f46f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:10.711330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360bb098-1614-4894-ae5c-dbc6997f46f4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:10.711925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360bb098-1614-4894-ae5c-dbc6997f46f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:10.714661 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360bb098-1614-4894-ae5c-dbc6997f46f4 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:10.724967 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 87/173] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:10.732652 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360bb098-1614-4894-ae5c-dbc6997f46f4 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:10.755841 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:10.872370 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360bb098-1614-4894-ae5c-dbc6997f46f4 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:10.886750 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2676cbc-126b-4381-8587-b5f402c98b28 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:10.914035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34102db0-f45b-4a0d-a04f-3ceebfd2e61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:10.914771 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34102db0-f45b-4a0d-a04f-3ceebfd2e61d None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:10.915149 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34102db0-f45b-4a0d-a04f-3ceebfd2e61d None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:10.915982 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34102db0-f45b-4a0d-a04f-3ceebfd2e61d None admin] PATH_INFO: `/v3/users/9a89bd577d164d69948fbb5e8b571169` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:10.917499 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34102db0-f45b-4a0d-a04f-3ceebfd2e61d None admin] RBAC: Authorizing `identity:get_user(user_id=9a89bd577d164d69948fbb5e8b571169)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:10.919240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34102db0-f45b-4a0d-a04f-3ceebfd2e61d None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:10.921442 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 88/174] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:10 2022] GET /identity/v3/users/9a89bd577d164d69948fbb5e8b571169 => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:10.929025 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34102db0-f45b-4a0d-a04f-3ceebfd2e61d None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:10.984224 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39997c37-1558-4024-93bd-bb4eff6396ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:10.985732 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39997c37-1558-4024-93bd-bb4eff6396ef None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:10.986032 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39997c37-1558-4024-93bd-bb4eff6396ef None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:10.986314 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39997c37-1558-4024-93bd-bb4eff6396ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:11.051892 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:11.068333 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34102db0-f45b-4a0d-a04f-3ceebfd2e61d None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:11.074125 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:11.110640 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd14e8b9-aac0-42de-8c21-dd0c2ec88c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:11.111421 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd14e8b9-aac0-42de-8c21-dd0c2ec88c99 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:11.111705 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd14e8b9-aac0-42de-8c21-dd0c2ec88c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:11.111976 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd14e8b9-aac0-42de-8c21-dd0c2ec88c99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:11.149839 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39997c37-1558-4024-93bd-bb4eff6396ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:11.151776 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39997c37-1558-4024-93bd-bb4eff6396ef None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:11.191895 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd14e8b9-aac0-42de-8c21-dd0c2ec88c99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:11.193587 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd14e8b9-aac0-42de-8c21-dd0c2ec88c99 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:11.316973 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:11.328146 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 87/175] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 672 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:11.335277 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9b2d5-e58a-4800-bf71-124a8af96149 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:11.335453 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 89/176] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:11.336032 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9b2d5-e58a-4800-bf71-124a8af96149 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:11.337224 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc08342-9b09-4c2a-9793-eaf0495eb2c8 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:11.337224 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc08342-9b09-4c2a-9793-eaf0495eb2c8 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:11.337224 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc08342-9b09-4c2a-9793-eaf0495eb2c8 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:11.337877 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9b2d5-e58a-4800-bf71-124a8af96149 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:11.338271 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 90/177] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:11.339567 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc08342-9b09-4c2a-9793-eaf0495eb2c8 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:11.342779 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 88/178] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:11 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:11.343825 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9b2d5-e58a-4800-bf71-124a8af96149 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:11.441172 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:11.475059 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc08342-9b09-4c2a-9793-eaf0495eb2c8 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:11.479309 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9b2d5-e58a-4800-bf71-124a8af96149 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:11.515349 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd853e-1c4a-48b9-a9a0-c9bdeca93765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:11.516990 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd853e-1c4a-48b9-a9a0-c9bdeca93765 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:11.517505 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd853e-1c4a-48b9-a9a0-c9bdeca93765 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:11.518061 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd853e-1c4a-48b9-a9a0-c9bdeca93765 None admin] PATH_INFO: `/v3/projects/f542d45b34d849d2aad08bc75ebc275a` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:11.519569 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd853e-1c4a-48b9-a9a0-c9bdeca93765 None admin] RBAC: Authorizing `identity:get_project(project_id=f542d45b34d849d2aad08bc75ebc275a)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:11.521447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd853e-1c4a-48b9-a9a0-c9bdeca93765 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:11.523877 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 91/179] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:11 2022] GET /identity/v3/projects/f542d45b34d849d2aad08bc75ebc275a => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:11.524328 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed56064-ea54-40e7-9368-3ab1cfe98df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:11.525000 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed56064-ea54-40e7-9368-3ab1cfe98df8 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:11.525284 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed56064-ea54-40e7-9368-3ab1cfe98df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:11.525817 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed56064-ea54-40e7-9368-3ab1cfe98df8 None admin] PATH_INFO: `/v3/projects/ea72a5b1e48a4077bac98d52c5077d43` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:11.526660 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f847ef2-fac2-422b-8533-6e75d68a5175 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:11.527008 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed56064-ea54-40e7-9368-3ab1cfe98df8 None admin] RBAC: Authorizing `identity:get_project(project_id=ea72a5b1e48a4077bac98d52c5077d43)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:11.527435 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f847ef2-fac2-422b-8533-6e75d68a5175 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:11.527901 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f847ef2-fac2-422b-8533-6e75d68a5175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:11.528623 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed56064-ea54-40e7-9368-3ab1cfe98df8 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:11.530395 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 89/180] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:11 2022] GET /identity/v3/projects/ea72a5b1e48a4077bac98d52c5077d43 => generated 312 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:11.531577 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230866f-204d-40e1-a15e-357aed2f427b None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:11.532548 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230866f-204d-40e1-a15e-357aed2f427b None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:11.532982 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230866f-204d-40e1-a15e-357aed2f427b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:11.549114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-6f847ef2-fac2-422b-8533-6e75d68a5175 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:11.550707 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-6f847ef2-fac2-422b-8533-6e75d68a5175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:11.555817 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-5230866f-204d-40e1-a15e-357aed2f427b None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:11.557063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-5230866f-204d-40e1-a15e-357aed2f427b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:11.620294 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:11.642280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 92/181] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:11.645738 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f847ef2-fac2-422b-8533-6e75d68a5175 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:11.656048 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 90/182] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:11.663543 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230866f-204d-40e1-a15e-357aed2f427b None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:11.745178 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:11.768304 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f847ef2-fac2-422b-8533-6e75d68a5175 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:11.778584 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230866f-204d-40e1-a15e-357aed2f427b None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:11.810076 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5307d6-e7b2-45e1-b74f-069c4861afd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:11.810695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5307d6-e7b2-45e1-b74f-069c4861afd1 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:11.810972 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5307d6-e7b2-45e1-b74f-069c4861afd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:11.811244 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5307d6-e7b2-45e1-b74f-069c4861afd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:11.812042 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5307d6-e7b2-45e1-b74f-069c4861afd1 None admin] RBAC: Adding query filter params (role.id=fbd7c81b4fba4b6d9e61b1e9bb3e2046, scope.project.id=ea72a5b1e48a4077bac98d52c5077d43, user.id=9a89bd577d164d69948fbb5e8b571169) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:11.812358 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5307d6-e7b2-45e1-b74f-069c4861afd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:11.813516 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c785a616-d1d0-4f70-bed6-5d10256d41d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:11.814132 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785a616-d1d0-4f70-bed6-5d10256d41d3 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:11.814394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5307d6-e7b2-45e1-b74f-069c4861afd1 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:11.814483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785a616-d1d0-4f70-bed6-5d10256d41d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:11.814764 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785a616-d1d0-4f70-bed6-5d10256d41d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:11.815512 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c785a616-d1d0-4f70-bed6-5d10256d41d3 None admin] RBAC: Adding query filter params (role.id=0eafd8a7ba2d47faaf5aecbb3360143b, scope.project.id=f542d45b34d849d2aad08bc75ebc275a, user.id=9a89bd577d164d69948fbb5e8b571169) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:11.815811 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c785a616-d1d0-4f70-bed6-5d10256d41d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:11.817297 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c785a616-d1d0-4f70-bed6-5d10256d41d3 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:11.820490 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 93/183] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:49:11 2022] GET /identity/v3/role_assignments?user.id=9a89bd577d164d69948fbb5e8b571169&scope.project.id=ea72a5b1e48a4077bac98d52c5077d43&role.id=fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:11.821787 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc8fd1-5449-495c-9390-ecca2747cf72 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:11.822074 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc8fd1-5449-495c-9390-ecca2747cf72 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:11.822349 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc8fd1-5449-495c-9390-ecca2747cf72 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:11.823572 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 94/184] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:11.828090 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc8fd1-5449-495c-9390-ecca2747cf72 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:11.828205 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 91/185] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:49:11 2022] GET /identity/v3/role_assignments?user.id=9a89bd577d164d69948fbb5e8b571169&scope.project.id=f542d45b34d849d2aad08bc75ebc275a&role.id=0eafd8a7ba2d47faaf5aecbb3360143b => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:11.828205 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c785a616-d1d0-4f70-bed6-5d10256d41d3 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:11.888268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:11.960262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc8fd1-5449-495c-9390-ecca2747cf72 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:11.982168 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c785a616-d1d0-4f70-bed6-5d10256d41d3 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:12.003334 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0327db3-a809-4c82-8eb4-3fdaccab6538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:12.004013 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0327db3-a809-4c82-8eb4-3fdaccab6538 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:12.004905 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0327db3-a809-4c82-8eb4-3fdaccab6538 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:12.005017 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0327db3-a809-4c82-8eb4-3fdaccab6538 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:12.018423 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0327db3-a809-4c82-8eb4-3fdaccab6538 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:12.024989 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-b0327db3-a809-4c82-8eb4-3fdaccab6538 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:49:12.024989 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:12.024989 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:12.024989 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:12.024989 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:12.024989 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:12.024989 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:12.024989 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:12.024989 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:12.024989 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:12.024989 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:12.024989 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:12.024989 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:12.024989 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:12.024989 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:12.024989 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:12.026203 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:12.026203 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:12.026203 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:12.026203 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:12.026203 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:12.026203 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:12.026203 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:12.026203 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:12.026203 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:12.026203 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:12.026203 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:12.026203 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:12.026203 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:12.026203 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:12.026203 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:12.026203 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:12.027001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:12.027001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:12.027001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:12.027001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:12.027001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:12.027001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:12.027001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:12.027001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:12.027001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:12.027001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:12.027001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:49:12.027001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:49:12.027001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 95/186] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:49:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:12.029872 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe35bb3-2adb-4d6a-a337-b164d8abffb3 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:12.030351 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe35bb3-2adb-4d6a-a337-b164d8abffb3 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:12.030808 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe35bb3-2adb-4d6a-a337-b164d8abffb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:12.039413 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6811b5-a11b-4c88-ae35-50f66fb50cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:12.040077 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6811b5-a11b-4c88-ae35-50f66fb50cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:12.040364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6811b5-a11b-4c88-ae35-50f66fb50cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:12.040665 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6811b5-a11b-4c88-ae35-50f66fb50cb3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:12.054102 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:12.069980 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-2fe35bb3-2adb-4d6a-a337-b164d8abffb3 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:12.071440 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6811b5-a11b-4c88-ae35-50f66fb50cb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:12.075908 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-2fe35bb3-2adb-4d6a-a337-b164d8abffb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:12.082291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-2a6811b5-a11b-4c88-ae35-50f66fb50cb3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:12.082291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:12.082291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:12.082291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:12.082291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:12.082291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:12.082291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:12.082291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:12.082291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:12.082291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:12.082291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:12.082291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:12.082291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:12.082291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:12.082291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:12.082291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:12.083264 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:12.083264 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:12.083264 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:12.083264 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:12.083264 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:12.083264 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:12.083264 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:12.083264 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:12.083264 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:12.083264 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:12.083264 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:12.083264 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:12.083264 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:12.083264 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:12.083264 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:12.083264 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:12.084175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:12.084175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:12.084175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:12.084175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:12.084175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:12.084175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:12.084175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:12.084175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:12.084175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:12.084175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:12.084175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:12.084175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:12.086975 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 92/187] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Aug 5 19:49:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:12.088103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6811b5-a11b-4c88-ae35-50f66fb50cb3 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:12.194631 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 96/188] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:12.195507 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe35bb3-2adb-4d6a-a337-b164d8abffb3 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:12.234111 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6811b5-a11b-4c88-ae35-50f66fb50cb3 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:12.252491 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:12.286044 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afb895-7cb6-44eb-bc06-3cf44e5816b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:12.286601 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afb895-7cb6-44eb-bc06-3cf44e5816b9 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:12.286880 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afb895-7cb6-44eb-bc06-3cf44e5816b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:12.287318 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afb895-7cb6-44eb-bc06-3cf44e5816b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:12.290305 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27afb895-7cb6-44eb-bc06-3cf44e5816b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:12.290305 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27afb895-7cb6-44eb-bc06-3cf44e5816b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:12.290564 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27afb895-7cb6-44eb-bc06-3cf44e5816b9 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:12.302207 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 93/189] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:12.303255 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afb895-7cb6-44eb-bc06-3cf44e5816b9 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:12.319279 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe35bb3-2adb-4d6a-a337-b164d8abffb3 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:12.356565 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb6799-2c85-4861-b1e1-9b2c1c8e9026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:12.357209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb6799-2c85-4861-b1e1-9b2c1c8e9026 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:12.357493 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb6799-2c85-4861-b1e1-9b2c1c8e9026 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:12.357767 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb6799-2c85-4861-b1e1-9b2c1c8e9026 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:12.358485 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebb6799-2c85-4861-b1e1-9b2c1c8e9026 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:12.358800 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebb6799-2c85-4861-b1e1-9b2c1c8e9026 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:12.360218 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebb6799-2c85-4861-b1e1-9b2c1c8e9026 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:12.368990 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 97/190] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:12.369657 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb6799-2c85-4861-b1e1-9b2c1c8e9026 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:12.375777 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:12.422015 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afb895-7cb6-44eb-bc06-3cf44e5816b9 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:12.461931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1df366-1adb-44e2-8f67-94247421a720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:12.462561 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1df366-1adb-44e2-8f67-94247421a720 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:12.462847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1df366-1adb-44e2-8f67-94247421a720 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:12.463112 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1df366-1adb-44e2-8f67-94247421a720 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:12.464249 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1df366-1adb-44e2-8f67-94247421a720 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:12.465719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1df366-1adb-44e2-8f67-94247421a720 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:12.467436 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 94/191] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:12.472788 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1df366-1adb-44e2-8f67-94247421a720 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:12.473717 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:12.477119 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb6799-2c85-4861-b1e1-9b2c1c8e9026 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:12.515067 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc49ffa-9c39-49fe-8f1b-fc0f15057464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:12.515708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc49ffa-9c39-49fe-8f1b-fc0f15057464 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:12.516028 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc49ffa-9c39-49fe-8f1b-fc0f15057464 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:12.516359 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc49ffa-9c39-49fe-8f1b-fc0f15057464 None admin] PATH_INFO: `/v3/users/71a49e6aceb747ba9f57bea5d9f9cd9c` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:12.530786 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc49ffa-9c39-49fe-8f1b-fc0f15057464 None admin] RBAC: Authorizing `identity:get_user(user_id=71a49e6aceb747ba9f57bea5d9f9cd9c)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:12.532391 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc49ffa-9c39-49fe-8f1b-fc0f15057464 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:12.534230 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 98/192] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:12 2022] GET /identity/v3/users/71a49e6aceb747ba9f57bea5d9f9cd9c => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:12.534892 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc49ffa-9c39-49fe-8f1b-fc0f15057464 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:12.582069 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1df366-1adb-44e2-8f67-94247421a720 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:12.612351 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:12.622486 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f2ba3-e79e-4b82-9962-a136804aa3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:12.623223 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f2ba3-e79e-4b82-9962-a136804aa3c1 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:12.623643 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f2ba3-e79e-4b82-9962-a136804aa3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:12.623924 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f2ba3-e79e-4b82-9962-a136804aa3c1 None admin] PATH_INFO: `/v3/users/71a49e6aceb747ba9f57bea5d9f9cd9c` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:12.625155 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f2ba3-e79e-4b82-9962-a136804aa3c1 None admin] RBAC: Authorizing `identity:get_user(user_id=71a49e6aceb747ba9f57bea5d9f9cd9c)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:12.626898 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f2ba3-e79e-4b82-9962-a136804aa3c1 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:12.632841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 95/193] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:12 2022] GET /identity/v3/users/71a49e6aceb747ba9f57bea5d9f9cd9c => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:12.632841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f2ba3-e79e-4b82-9962-a136804aa3c1 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:12.644990 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc49ffa-9c39-49fe-8f1b-fc0f15057464 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:12.681455 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96abb9ce-ddad-4630-9565-9233164502d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:12.682066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96abb9ce-ddad-4630-9565-9233164502d5 None admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:12.682343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96abb9ce-ddad-4630-9565-9233164502d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:12.682614 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96abb9ce-ddad-4630-9565-9233164502d5 None admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:12.683292 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96abb9ce-ddad-4630-9565-9233164502d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:12.684772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96abb9ce-ddad-4630-9565-9233164502d5 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:12.713643 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 99/194] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Aug 5 19:49:12 2022] POST /identity/v3/users => generated 297 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:12.714777 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96abb9ce-ddad-4630-9565-9233164502d5 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:12.733535 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:12.739558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f2ba3-e79e-4b82-9962-a136804aa3c1 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:12.783041 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46149723-4452-4c9b-9237-ac0889241178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:12.783988 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46149723-4452-4c9b-9237-ac0889241178 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:12.784269 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46149723-4452-4c9b-9237-ac0889241178 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:12.784543 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46149723-4452-4c9b-9237-ac0889241178 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:12.853232 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96abb9ce-ddad-4630-9565-9233164502d5 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:12.861650 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:12.871369 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46149723-4452-4c9b-9237-ac0889241178 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:12.873506 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46149723-4452-4c9b-9237-ac0889241178 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:12.897054 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde7c350-0900-4914-b9f4-2dbfa297c98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:12.897685 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7c350-0900-4914-b9f4-2dbfa297c98a None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:12.897965 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7c350-0900-4914-b9f4-2dbfa297c98a None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:12.898888 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7c350-0900-4914-b9f4-2dbfa297c98a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:12.990200 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde7c350-0900-4914-b9f4-2dbfa297c98a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:12.991686 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde7c350-0900-4914-b9f4-2dbfa297c98a None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:13.021770 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 96/195] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:13.023551 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8c8e4-03bf-476b-a992-d36f4aa0e48e None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:13.024035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8c8e4-03bf-476b-a992-d36f4aa0e48e None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:13.024530 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8c8e4-03bf-476b-a992-d36f4aa0e48e None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:13.025078 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:13.026728 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 97/196] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:13.027533 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8c8e4-03bf-476b-a992-d36f4aa0e48e None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:13.129699 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 100/197] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:13.131034 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762c317-e9c4-4dc0-8583-18da2cd9a106 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:13.131323 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762c317-e9c4-4dc0-8583-18da2cd9a106 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:13.131597 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762c317-e9c4-4dc0-8583-18da2cd9a106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:13.155006 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-2762c317-e9c4-4dc0-8583-18da2cd9a106 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:13.156001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-2762c317-e9c4-4dc0-8583-18da2cd9a106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:13.161573 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:13.169957 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8c8e4-03bf-476b-a992-d36f4aa0e48e None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:13.210803 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a9d9e-a8bd-4407-9d86-1711f677db82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:13.211824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a9d9e-a8bd-4407-9d86-1711f677db82 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:13.212294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a9d9e-a8bd-4407-9d86-1711f677db82 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:13.212804 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a9d9e-a8bd-4407-9d86-1711f677db82 None admin] PATH_INFO: `/v3/projects/c0ded24d8148467b9a56fa55332feb56` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:13.214284 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a9d9e-a8bd-4407-9d86-1711f677db82 None admin] RBAC: Authorizing `identity:get_project(project_id=c0ded24d8148467b9a56fa55332feb56)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:13.215946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a9d9e-a8bd-4407-9d86-1711f677db82 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:13.218312 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 98/198] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:13 2022] GET /identity/v3/projects/c0ded24d8148467b9a56fa55332feb56 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:13.220035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a9d9e-a8bd-4407-9d86-1711f677db82 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:13.240427 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 101/199] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:13.249286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762c317-e9c4-4dc0-8583-18da2cd9a106 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:13.328993 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:13.329975 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a9d9e-a8bd-4407-9d86-1711f677db82 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:13.377956 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8455157-32a3-49f1-8466-db003959e9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:13.379037 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8455157-32a3-49f1-8466-db003959e9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:13.379459 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8455157-32a3-49f1-8466-db003959e9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:13.380000 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8455157-32a3-49f1-8466-db003959e9b6 None admin] PATH_INFO: `/v3/projects/c0ded24d8148467b9a56fa55332feb56` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:13.381706 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8455157-32a3-49f1-8466-db003959e9b6 None admin] RBAC: Authorizing `identity:get_project(project_id=c0ded24d8148467b9a56fa55332feb56)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:13.383620 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8455157-32a3-49f1-8466-db003959e9b6 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:13.386273 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 99/200] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:13 2022] GET /identity/v3/projects/c0ded24d8148467b9a56fa55332feb56 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:13.392766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8455157-32a3-49f1-8466-db003959e9b6 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:13.392935 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762c317-e9c4-4dc0-8583-18da2cd9a106 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:13.434809 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bffa338-38a5-4a16-8253-018830e5b0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:13.435913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffa338-38a5-4a16-8253-018830e5b0f0 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:13.436394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffa338-38a5-4a16-8253-018830e5b0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:13.436935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:13.437351 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffa338-38a5-4a16-8253-018830e5b0f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:13.439168 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bffa338-38a5-4a16-8253-018830e5b0f0 None admin] RBAC: Adding query filter params (role.id=89919f784ad144529b07c8ab92ee7d48, scope.project.id=c0ded24d8148467b9a56fa55332feb56, user.id=71a49e6aceb747ba9f57bea5d9f9cd9c) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:13.439736 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bffa338-38a5-4a16-8253-018830e5b0f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:13.441443 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bffa338-38a5-4a16-8253-018830e5b0f0 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:13.450672 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 102/201] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:49:13 2022] GET /identity/v3/role_assignments?user.id=71a49e6aceb747ba9f57bea5d9f9cd9c&scope.project.id=c0ded24d8148467b9a56fa55332feb56&role.id=89919f784ad144529b07c8ab92ee7d48 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:13.451939 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bffa338-38a5-4a16-8253-018830e5b0f0 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:13.587576 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8455157-32a3-49f1-8466-db003959e9b6 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:13.639604 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3225a8-b337-465c-83a8-ce2f7cd14676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:13.640228 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3225a8-b337-465c-83a8-ce2f7cd14676 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:13.640475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3225a8-b337-465c-83a8-ce2f7cd14676 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:13.640811 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3225a8-b337-465c-83a8-ce2f7cd14676 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:13.652899 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bffa338-38a5-4a16-8253-018830e5b0f0 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:13.662930 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3225a8-b337-465c-83a8-ce2f7cd14676 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:13.664541 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:13.669983 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-0f3225a8-b337-465c-83a8-ce2f7cd14676 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:49:13.669983 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:13.669983 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:13.669983 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:13.669983 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:13.669983 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:13.669983 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:13.669983 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:13.669983 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:13.669983 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:13.669983 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:13.669983 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:13.669983 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:13.669983 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:13.669983 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:13.669983 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:13.670937 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:13.670937 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:13.670937 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:13.670937 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:13.670937 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:13.670937 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:13.670937 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:13.670937 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:13.670937 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:13.670937 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:13.670937 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:13.670937 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:13.670937 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:13.670937 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:13.670937 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:13.670937 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:13.671715 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:13.671715 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:13.671715 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:13.671715 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:13.671715 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:13.671715 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:13.671715 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:13.671715 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:13.671715 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:13.671715 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:13.671715 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:49:13.671715 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:13.675507 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 100/202] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:13.681581 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3225a8-b337-465c-83a8-ce2f7cd14676 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:13.712700 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16eea9-58b0-479b-ada5-01c4069bf105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:13.713839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16eea9-58b0-479b-ada5-01c4069bf105 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:13.716614 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16eea9-58b0-479b-ada5-01c4069bf105 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:13.717064 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16eea9-58b0-479b-ada5-01c4069bf105 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:13.718259 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea16eea9-58b0-479b-ada5-01c4069bf105 None admin] RBAC: Adding query filter params (role.id=0eafd8a7ba2d47faaf5aecbb3360143b, scope.project.id=c0ded24d8148467b9a56fa55332feb56, user.id=71a49e6aceb747ba9f57bea5d9f9cd9c) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:13.720124 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea16eea9-58b0-479b-ada5-01c4069bf105 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:13.723960 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea16eea9-58b0-479b-ada5-01c4069bf105 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:13.734707 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 103/203] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:49:13 2022] GET /identity/v3/role_assignments?user.id=71a49e6aceb747ba9f57bea5d9f9cd9c&scope.project.id=c0ded24d8148467b9a56fa55332feb56&role.id=0eafd8a7ba2d47faaf5aecbb3360143b => generated 259 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:13.850828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:13.867077 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3225a8-b337-465c-83a8-ce2f7cd14676 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:13.918632 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e86d6-82bd-403b-ab58-a9e25f0f22bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:13.919178 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e86d6-82bd-403b-ab58-a9e25f0f22bc None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:13.919460 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e86d6-82bd-403b-ab58-a9e25f0f22bc None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:13.919736 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e86d6-82bd-403b-ab58-a9e25f0f22bc None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:13.920465 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e86d6-82bd-403b-ab58-a9e25f0f22bc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:13.920803 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e86d6-82bd-403b-ab58-a9e25f0f22bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:13.922848 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e86d6-82bd-403b-ab58-a9e25f0f22bc None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:13.938697 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 101/204] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:13.946623 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16eea9-58b0-479b-ada5-01c4069bf105 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:14.095483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16eea9-58b0-479b-ada5-01c4069bf105 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:14.146083 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77715249-93a9-43cc-b0a2-f68eebfae586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:14.147676 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77715249-93a9-43cc-b0a2-f68eebfae586 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:14.147676 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77715249-93a9-43cc-b0a2-f68eebfae586 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:14.147676 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77715249-93a9-43cc-b0a2-f68eebfae586 None admin] PATH_INFO: `/v3/users/246108d952bf421497f9b844b50802ff` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:14.150722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77715249-93a9-43cc-b0a2-f68eebfae586 None admin] RBAC: Authorizing `identity:get_user(user_id=246108d952bf421497f9b844b50802ff)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:14.152226 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77715249-93a9-43cc-b0a2-f68eebfae586 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:14.154106 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 104/205] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:13 2022] GET /identity/v3/users/246108d952bf421497f9b844b50802ff => generated 289 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:14.160208 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e86d6-82bd-403b-ab58-a9e25f0f22bc None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:14.283679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e86d6-82bd-403b-ab58-a9e25f0f22bc None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:14.319604 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4c187-2b3b-49fd-bd17-025430179535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:14.320225 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4c187-2b3b-49fd-bd17-025430179535 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:14.320505 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4c187-2b3b-49fd-bd17-025430179535 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:14.320810 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4c187-2b3b-49fd-bd17-025430179535 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:14.345552 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:14.393488 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a4c187-2b3b-49fd-bd17-025430179535 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:14.394955 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a4c187-2b3b-49fd-bd17-025430179535 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:14.509874 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 102/206] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:14.515545 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77715249-93a9-43cc-b0a2-f68eebfae586 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:14.596301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:14.680427 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77715249-93a9-43cc-b0a2-f68eebfae586 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:14.731665 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1074ee7b-9fa2-49af-8aff-d5f5275979d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:14.732324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074ee7b-9fa2-49af-8aff-d5f5275979d5 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:14.732629 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074ee7b-9fa2-49af-8aff-d5f5275979d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:14.732916 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074ee7b-9fa2-49af-8aff-d5f5275979d5 None admin] PATH_INFO: `/v3/projects/f542d45b34d849d2aad08bc75ebc275a` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:14.734171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1074ee7b-9fa2-49af-8aff-d5f5275979d5 None admin] RBAC: Authorizing `identity:get_project(project_id=f542d45b34d849d2aad08bc75ebc275a)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:14.736454 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1074ee7b-9fa2-49af-8aff-d5f5275979d5 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:14.738524 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 105/207] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:14 2022] GET /identity/v3/projects/f542d45b34d849d2aad08bc75ebc275a => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:14.744606 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4c187-2b3b-49fd-bd17-025430179535 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:14.869732 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:14.946388 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4c187-2b3b-49fd-bd17-025430179535 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:15.013322 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7989630-3076-47e8-b070-1b62e0d4771f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:15.013958 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7989630-3076-47e8-b070-1b62e0d4771f None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:15.014242 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7989630-3076-47e8-b070-1b62e0d4771f None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:15.014517 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7989630-3076-47e8-b070-1b62e0d4771f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:15.015331 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7989630-3076-47e8-b070-1b62e0d4771f None admin] RBAC: Adding query filter params (role.id=9de09806c19c49e7aba24d414a65eb5f, scope.project.id=f542d45b34d849d2aad08bc75ebc275a, user.id=246108d952bf421497f9b844b50802ff) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:15.015642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7989630-3076-47e8-b070-1b62e0d4771f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:15.017552 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7989630-3076-47e8-b070-1b62e0d4771f None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:15.024090 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 103/208] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:49:14 2022] GET /identity/v3/role_assignments?user.id=246108d952bf421497f9b844b50802ff&scope.project.id=f542d45b34d849d2aad08bc75ebc275a&role.id=9de09806c19c49e7aba24d414a65eb5f => generated 608 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:15.222861 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3346981-8a8d-4535-be6e-ea87e3d3cfde None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:15.224255 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3346981-8a8d-4535-be6e-ea87e3d3cfde None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:15.225039 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3346981-8a8d-4535-be6e-ea87e3d3cfde None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:15.228128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 106/209] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:15.237241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce6735-b772-495a-97c3-14804cb23147 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:15.238161 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce6735-b772-495a-97c3-14804cb23147 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:15.238885 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce6735-b772-495a-97c3-14804cb23147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:15.282698 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-72ce6735-b772-495a-97c3-14804cb23147 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:15.285173 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-72ce6735-b772-495a-97c3-14804cb23147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:15.363438 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788dd1e-7e9b-4095-96bb-9d3b76b9a786 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:15.364354 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788dd1e-7e9b-4095-96bb-9d3b76b9a786 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:15.365092 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788dd1e-7e9b-4095-96bb-9d3b76b9a786 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:15.367785 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 107/210] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:49:15.373460 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287c638-896a-48a3-8537-14fc730b577c None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:15.373794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287c638-896a-48a3-8537-14fc730b577c None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:15.374020 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287c638-896a-48a3-8537-14fc730b577c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:15.400890 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-e287c638-896a-48a3-8537-14fc730b577c None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:15.402068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e287c638-896a-48a3-8537-14fc730b577c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:15.409746 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 104/211] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:15.459410 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ce6735-b772-495a-97c3-14804cb23147 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:15.486260 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 108/212] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:15.519420 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e287c638-896a-48a3-8537-14fc730b577c None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:15.550070 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:15.567501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:15.653526 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e287c638-896a-48a3-8537-14fc730b577c None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:15.674745 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ce6735-b772-495a-97c3-14804cb23147 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:15.695170 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ce6ad-e459-4f1c-a0be-9f78442b5d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:15.696302 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ce6ad-e459-4f1c-a0be-9f78442b5d28 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:15.696804 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ce6ad-e459-4f1c-a0be-9f78442b5d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:15.697558 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ce6ad-e459-4f1c-a0be-9f78442b5d28 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:15.711345 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ce6ad-e459-4f1c-a0be-9f78442b5d28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:15.717680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-7f4ce6ad-e459-4f1c-a0be-9f78442b5d28 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:15.717680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:15.717680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:15.717680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:15.717680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:15.717680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:15.717680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:15.717680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:15.717680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:15.717680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:15.717680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:15.717680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:15.717680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:15.717680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:15.717680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:15.717680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:15.718571 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:15.718571 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:15.718571 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:15.718571 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:15.718571 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:15.718571 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:15.718571 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:15.718571 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:15.718571 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:15.718571 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:15.718571 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:15.718571 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:15.718571 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:15.718571 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:15.718571 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:15.718571 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:15.720630 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:15.720630 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:15.720630 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:15.720630 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:15.720630 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:15.720630 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:15.720630 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:15.720630 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:15.720630 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:15.720630 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:15.720630 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:15.720630 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:49:15.722243 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 109/213] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:15.728930 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ce6ad-e459-4f1c-a0be-9f78442b5d28 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:15.730079 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ecb15-472a-4784-a41e-8ff1e8de74b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:15.730948 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ecb15-472a-4784-a41e-8ff1e8de74b7 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:15.731444 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ecb15-472a-4784-a41e-8ff1e8de74b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:15.731910 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ecb15-472a-4784-a41e-8ff1e8de74b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:15.734579 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ecb15-472a-4784-a41e-8ff1e8de74b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:15.737017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ecb15-472a-4784-a41e-8ff1e8de74b7 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:15.739487 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 105/214] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:15.745112 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ecb15-472a-4784-a41e-8ff1e8de74b7 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:15.865035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:15.871584 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ce6ad-e459-4f1c-a0be-9f78442b5d28 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:15.882786 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ecb15-472a-4784-a41e-8ff1e8de74b7 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:15.901720 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:15.927050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7412799-984e-4b74-b0e1-4031eb64cc9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:15.927444 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504494df-badc-453d-8186-ae29125b5d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:15.928083 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504494df-badc-453d-8186-ae29125b5d59 None admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:15.928364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504494df-badc-453d-8186-ae29125b5d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:15.928691 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504494df-badc-453d-8186-ae29125b5d59 None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:15.929047 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7412799-984e-4b74-b0e1-4031eb64cc9a None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:15.929396 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504494df-badc-453d-8186-ae29125b5d59 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:15.929725 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7412799-984e-4b74-b0e1-4031eb64cc9a None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:15.930192 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7412799-984e-4b74-b0e1-4031eb64cc9a None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:15.930955 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504494df-badc-453d-8186-ae29125b5d59 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:15.931361 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7412799-984e-4b74-b0e1-4031eb64cc9a None admin] RBAC: Adding query filter params (name=member) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:15.931859 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7412799-984e-4b74-b0e1-4031eb64cc9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:15.933564 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7412799-984e-4b74-b0e1-4031eb64cc9a None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:15.946455 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 110/215] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:15.954790 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7412799-984e-4b74-b0e1-4031eb64cc9a None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:15.962095 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 106/216] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Aug 5 19:49:15 2022] POST /identity/v3/users => generated 297 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:16.088391 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7412799-984e-4b74-b0e1-4031eb64cc9a None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:16.135902 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52118b4a-1ca7-48b7-a01f-f6aad2ec7cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:16.136693 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52118b4a-1ca7-48b7-a01f-f6aad2ec7cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:16.136993 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52118b4a-1ca7-48b7-a01f-f6aad2ec7cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:16.137482 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52118b4a-1ca7-48b7-a01f-f6aad2ec7cd3 None admin] PATH_INFO: `/v3/users/9a522bb1b075409dbbcc974adfa48476` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:16.161291 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52118b4a-1ca7-48b7-a01f-f6aad2ec7cd3 None admin] RBAC: Authorizing `identity:get_user(user_id=9a522bb1b075409dbbcc974adfa48476)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:16.163590 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52118b4a-1ca7-48b7-a01f-f6aad2ec7cd3 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:16.166955 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 111/217] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:15 2022] GET /identity/v3/users/9a522bb1b075409dbbcc974adfa48476 => generated 297 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:16.173528 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504494df-badc-453d-8186-ae29125b5d59 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:16.323411 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504494df-badc-453d-8186-ae29125b5d59 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:16.387413 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc46199c-0780-4c4d-946b-43b4bb43095b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:16.388590 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46199c-0780-4c4d-946b-43b4bb43095b None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:16.389093 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46199c-0780-4c4d-946b-43b4bb43095b None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:16.389650 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46199c-0780-4c4d-946b-43b4bb43095b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:16.405967 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a2465-b144-4cdb-845a-5b34fef21e2c None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:16.407681 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a2465-b144-4cdb-845a-5b34fef21e2c None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:16.408161 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a2465-b144-4cdb-845a-5b34fef21e2c None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:16.410452 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 112/218] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:49:16.416430 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914587d-4e67-4b2f-843c-5bc3fed08824 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:16.416836 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914587d-4e67-4b2f-843c-5bc3fed08824 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:16.417147 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914587d-4e67-4b2f-843c-5bc3fed08824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:16.442744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-b914587d-4e67-4b2f-843c-5bc3fed08824 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:16.444043 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-b914587d-4e67-4b2f-843c-5bc3fed08824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:16.453268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:16.472840 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:16.498988 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc46199c-0780-4c4d-946b-43b4bb43095b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:16.501289 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc46199c-0780-4c4d-946b-43b4bb43095b None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:16.562371 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 113/219] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:16.588380 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914587d-4e67-4b2f-843c-5bc3fed08824 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:16.657440 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 107/220] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:16.668135 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc46199c-0780-4c4d-946b-43b4bb43095b None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:16.725623 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914587d-4e67-4b2f-843c-5bc3fed08824 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:16.773031 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:16.776541 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476e25b-f632-4cf2-8651-7409648730af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:16.778016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476e25b-f632-4cf2-8651-7409648730af None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:16.778492 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476e25b-f632-4cf2-8651-7409648730af None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:16.778977 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476e25b-f632-4cf2-8651-7409648730af None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:16.792735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476e25b-f632-4cf2-8651-7409648730af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:16.798793 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-b476e25b-f632-4cf2-8651-7409648730af None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:16.798793 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:16.798793 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:16.798793 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:16.798793 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:16.798793 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:16.798793 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:16.798793 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:16.798793 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:16.798793 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:16.798793 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:16.798793 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:16.798793 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:16.798793 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:16.798793 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:16.798793 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:16.799735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:16.799735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:16.799735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:16.799735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:16.799735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:16.799735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:16.799735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:16.799735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:16.799735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:16.799735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:16.799735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:16.799735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:16.799735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:16.799735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:16.799735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:16.799735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:16.800539 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:16.800539 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:16.800539 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:16.800539 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:16.800539 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:16.800539 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:16.800539 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:16.800539 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:16.800539 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:16.800539 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:16.800539 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:16.800539 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:49:16.800539 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 114/221] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Aug 5 19:49:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:16.802136 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a542da-3969-485a-b6dc-db2abd3ced6b None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:16.802478 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a542da-3969-485a-b6dc-db2abd3ced6b None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:16.802820 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a542da-3969-485a-b6dc-db2abd3ced6b None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:16.808449 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 115/222] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:16.809350 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a542da-3969-485a-b6dc-db2abd3ced6b None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:16.820710 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc46199c-0780-4c4d-946b-43b4bb43095b None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:16.822051 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:16.870540 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edb9c2-0933-496c-a4d3-b20b6eb47ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:16.871044 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edb9c2-0933-496c-a4d3-b20b6eb47ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:16.871324 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edb9c2-0933-496c-a4d3-b20b6eb47ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:16.871600 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edb9c2-0933-496c-a4d3-b20b6eb47ee0 None admin] PATH_INFO: `/v3/projects/c0ded24d8148467b9a56fa55332feb56` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:16.872837 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5edb9c2-0933-496c-a4d3-b20b6eb47ee0 None admin] RBAC: Authorizing `identity:get_project(project_id=c0ded24d8148467b9a56fa55332feb56)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:16.874362 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5edb9c2-0933-496c-a4d3-b20b6eb47ee0 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:16.881157 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 108/223] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:16 2022] GET /identity/v3/projects/c0ded24d8148467b9a56fa55332feb56 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:16.881157 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a3967-8c19-4835-a60c-a7441888ece9 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:16.881157 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a3967-8c19-4835-a60c-a7441888ece9 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:16.881157 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a3967-8c19-4835-a60c-a7441888ece9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:16.899814 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-9e9a3967-8c19-4835-a60c-a7441888ece9 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:16.901368 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-9e9a3967-8c19-4835-a60c-a7441888ece9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:16.950055 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a542da-3969-485a-b6dc-db2abd3ced6b None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:16.991744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f6516-1369-4372-b2f0-c0940119c013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:16.992374 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f6516-1369-4372-b2f0-c0940119c013 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:16.992717 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f6516-1369-4372-b2f0-c0940119c013 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:16.993066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f6516-1369-4372-b2f0-c0940119c013 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:16.993794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f6516-1369-4372-b2f0-c0940119c013 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:16.994159 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f6516-1369-4372-b2f0-c0940119c013 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:16.995637 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f6516-1369-4372-b2f0-c0940119c013 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:17.007176 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 116/224] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:17.008263 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f6516-1369-4372-b2f0-c0940119c013 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:17.026002 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 109/225] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:17.027548 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9a3967-8c19-4835-a60c-a7441888ece9 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:17.127331 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:17.171138 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f6516-1369-4372-b2f0-c0940119c013 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:17.173483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:17.184332 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9a3967-8c19-4835-a60c-a7441888ece9 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:17.210061 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da021db8-4986-42e3-b23e-7f4e1f148d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:17.210745 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da021db8-4986-42e3-b23e-7f4e1f148d85 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:17.211100 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da021db8-4986-42e3-b23e-7f4e1f148d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:17.211450 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da021db8-4986-42e3-b23e-7f4e1f148d85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:17.212416 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da021db8-4986-42e3-b23e-7f4e1f148d85 None admin] RBAC: Adding query filter params (role.id=fbd7c81b4fba4b6d9e61b1e9bb3e2046, scope.project.id=c0ded24d8148467b9a56fa55332feb56, user.id=9a522bb1b075409dbbcc974adfa48476) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:17.212898 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da021db8-4986-42e3-b23e-7f4e1f148d85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:17.214471 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da021db8-4986-42e3-b23e-7f4e1f148d85 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:17.222110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 117/226] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:49:17 2022] GET /identity/v3/role_assignments?user.id=9a522bb1b075409dbbcc974adfa48476&scope.project.id=c0ded24d8148467b9a56fa55332feb56&role.id=fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:17.223062 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da021db8-4986-42e3-b23e-7f4e1f148d85 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:17.226558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b7d5e6-3b31-4749-9658-b8b55a1fd1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:17.227534 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7d5e6-3b31-4749-9658-b8b55a1fd1cb None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:17.228024 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7d5e6-3b31-4749-9658-b8b55a1fd1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:17.228489 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7d5e6-3b31-4749-9658-b8b55a1fd1cb None admin] PATH_INFO: `/v3/users/71a49e6aceb747ba9f57bea5d9f9cd9c` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:17.230385 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b7d5e6-3b31-4749-9658-b8b55a1fd1cb None admin] RBAC: Authorizing `identity:get_user(user_id=71a49e6aceb747ba9f57bea5d9f9cd9c)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:17.232713 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b7d5e6-3b31-4749-9658-b8b55a1fd1cb None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:17.235323 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 110/227] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:17 2022] GET /identity/v3/users/71a49e6aceb747ba9f57bea5d9f9cd9c => generated 283 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:17.241137 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b7d5e6-3b31-4749-9658-b8b55a1fd1cb None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:17.360524 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da021db8-4986-42e3-b23e-7f4e1f148d85 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:17.374922 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b7d5e6-3b31-4749-9658-b8b55a1fd1cb None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:17.399890 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47f286-e98b-4e23-ae0f-b5729b86f6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:17.400868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47f286-e98b-4e23-ae0f-b5729b86f6b9 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:17.401430 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47f286-e98b-4e23-ae0f-b5729b86f6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:17.401716 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47f286-e98b-4e23-ae0f-b5729b86f6b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:17.403815 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:17.414401 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df47f286-e98b-4e23-ae0f-b5729b86f6b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:17.420011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-df47f286-e98b-4e23-ae0f-b5729b86f6b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:49:17.420011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:17.420011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:17.420011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:17.420011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:17.420011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:17.420011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:17.420011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:17.420011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:17.420011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:17.420011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:17.420011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:17.420011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:17.420011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:17.420011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:17.420011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:17.421925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:17.421925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:17.421925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:17.421925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:17.421925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:17.421925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:17.421925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:17.421925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:17.421925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:17.421925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:17.421925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:17.421925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:17.421925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:17.421925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:17.421925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:17.421925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:17.423465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:17.423465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:17.423465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:17.423465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:17.423465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:17.423465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:17.423465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:17.423465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:17.423465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:17.423465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:17.423465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:49:17.423465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:49:17.423465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 118/228] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:49:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:17.436547 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47f286-e98b-4e23-ae0f-b5729b86f6b9 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:17.456919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284252fd-0afc-4170-8623-feabb0bd77a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:17.457691 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284252fd-0afc-4170-8623-feabb0bd77a7 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:17.458037 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284252fd-0afc-4170-8623-feabb0bd77a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:17.458316 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284252fd-0afc-4170-8623-feabb0bd77a7 None admin] PATH_INFO: `/v3/projects/c0ded24d8148467b9a56fa55332feb56` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:17.460939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284252fd-0afc-4170-8623-feabb0bd77a7 None admin] RBAC: Authorizing `identity:get_project(project_id=c0ded24d8148467b9a56fa55332feb56)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:17.462501 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284252fd-0afc-4170-8623-feabb0bd77a7 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:17.468429 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 111/229] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:17 2022] GET /identity/v3/projects/c0ded24d8148467b9a56fa55332feb56 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:17.472022 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284252fd-0afc-4170-8623-feabb0bd77a7 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:17.478462 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:17.616674 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47f286-e98b-4e23-ae0f-b5729b86f6b9 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:17.641834 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284252fd-0afc-4170-8623-feabb0bd77a7 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:17.671093 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c4279-5fc9-4584-8ed8-baef7070210d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:17.671760 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c4279-5fc9-4584-8ed8-baef7070210d None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:17.672755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c4279-5fc9-4584-8ed8-baef7070210d None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:17.673128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c4279-5fc9-4584-8ed8-baef7070210d None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:17.673870 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c4279-5fc9-4584-8ed8-baef7070210d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:17.674185 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c4279-5fc9-4584-8ed8-baef7070210d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:17.675728 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c4279-5fc9-4584-8ed8-baef7070210d None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:17.681507 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ad26b-f8e2-409d-90d0-5d83144fe4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:17.682658 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ad26b-f8e2-409d-90d0-5d83144fe4c4 None admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:17.683140 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ad26b-f8e2-409d-90d0-5d83144fe4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:17.683646 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ad26b-f8e2-409d-90d0-5d83144fe4c4 None admin] PATH_INFO: `/v3/projects/c0ded24d8148467b9a56fa55332feb56/users/71a49e6aceb747ba9f57bea5d9f9cd9c/roles/0eafd8a7ba2d47faaf5aecbb3360143b` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:17.686229 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ad26b-f8e2-409d-90d0-5d83144fe4c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0ded24d8148467b9a56fa55332feb56, user_id=71a49e6aceb747ba9f57bea5d9f9cd9c, role_id=0eafd8a7ba2d47faaf5aecbb3360143b)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:17.688200 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ad26b-f8e2-409d-90d0-5d83144fe4c4 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:17.694754 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 119/230] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:17.704630 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c4279-5fc9-4584-8ed8-baef7070210d None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:17.713587 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 112/231] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Aug 5 19:49:17 2022] PUT /identity/v3/projects/c0ded24d8148467b9a56fa55332feb56/users/71a49e6aceb747ba9f57bea5d9f9cd9c/roles/0eafd8a7ba2d47faaf5aecbb3360143b => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:17.782691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:17.843565 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c4279-5fc9-4584-8ed8-baef7070210d None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:17.896425 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22115562-3d40-46dc-9740-098fa353f677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:17.897118 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22115562-3d40-46dc-9740-098fa353f677 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:17.897395 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22115562-3d40-46dc-9740-098fa353f677 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:17.897679 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22115562-3d40-46dc-9740-098fa353f677 None admin] PATH_INFO: `/v3/users/71a49e6aceb747ba9f57bea5d9f9cd9c` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:17.898877 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22115562-3d40-46dc-9740-098fa353f677 None admin] RBAC: Authorizing `identity:get_user(user_id=71a49e6aceb747ba9f57bea5d9f9cd9c)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:17.900609 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22115562-3d40-46dc-9740-098fa353f677 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:17.902979 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 120/232] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:17 2022] GET /identity/v3/users/71a49e6aceb747ba9f57bea5d9f9cd9c => generated 283 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:17.908951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ad26b-f8e2-409d-90d0-5d83144fe4c4 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:18.010881 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:18.067214 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ad26b-f8e2-409d-90d0-5d83144fe4c4 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:18.109156 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc665a1-b04e-48a0-99f9-de07b5ef1d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:18.110060 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc665a1-b04e-48a0-99f9-de07b5ef1d5f None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:18.110439 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc665a1-b04e-48a0-99f9-de07b5ef1d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:18.110823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc665a1-b04e-48a0-99f9-de07b5ef1d5f None admin] PATH_INFO: `/v3/projects/c0ded24d8148467b9a56fa55332feb56` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:18.121909 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc665a1-b04e-48a0-99f9-de07b5ef1d5f None admin] RBAC: Authorizing `identity:get_project(project_id=c0ded24d8148467b9a56fa55332feb56)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:18.123587 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc665a1-b04e-48a0-99f9-de07b5ef1d5f None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:18.126700 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 113/233] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:17 2022] GET /identity/v3/projects/c0ded24d8148467b9a56fa55332feb56 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:18.132294 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22115562-3d40-46dc-9740-098fa353f677 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:18.250844 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22115562-3d40-46dc-9740-098fa353f677 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:18.285304 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:18.288313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24bc882-4c79-4d42-99b1-744a2c2e4464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:18.289297 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bc882-4c79-4d42-99b1-744a2c2e4464 None admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:18.289819 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bc882-4c79-4d42-99b1-744a2c2e4464 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:18.290438 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bc882-4c79-4d42-99b1-744a2c2e4464 None admin] PATH_INFO: `/v3/projects/c0ded24d8148467b9a56fa55332feb56/users/71a49e6aceb747ba9f57bea5d9f9cd9c/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:18.292745 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24bc882-4c79-4d42-99b1-744a2c2e4464 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0ded24d8148467b9a56fa55332feb56, user_id=71a49e6aceb747ba9f57bea5d9f9cd9c, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:18.294493 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24bc882-4c79-4d42-99b1-744a2c2e4464 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:18.311794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 121/234] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Aug 5 19:49:18 2022] PUT /identity/v3/projects/c0ded24d8148467b9a56fa55332feb56/users/71a49e6aceb747ba9f57bea5d9f9cd9c/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:18.842487 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e0b06a-7704-44fc-8224-04c89b4a6b71 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:18.842763 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e0b06a-7704-44fc-8224-04c89b4a6b71 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:18.843278 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e0b06a-7704-44fc-8224-04c89b4a6b71 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:18.846087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 114/235] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:18.852048 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1516430-7cb5-46e6-822a-a12961d38762 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:18.852992 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1516430-7cb5-46e6-822a-a12961d38762 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:18.853468 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1516430-7cb5-46e6-822a-a12961d38762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:18.880069 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-d1516430-7cb5-46e6-822a-a12961d38762 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:18.881635 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-d1516430-7cb5-46e6-822a-a12961d38762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:18.976983 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 122/236] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:19.016121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e0b06a-7704-44fc-8224-04c89b4a6b71 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:19.128347 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ca7cb-28a6-4918-a9a9-a27fd969802d None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:19.128672 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ca7cb-28a6-4918-a9a9-a27fd969802d None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:19.128925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ca7cb-28a6-4918-a9a9-a27fd969802d None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:19.130286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 123/237] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:49:19.138882 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b55046-67c6-45f3-9d41-855cc5a36109 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:19.139383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b55046-67c6-45f3-9d41-855cc5a36109 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:19.139733 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b55046-67c6-45f3-9d41-855cc5a36109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:19.165828 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-85b55046-67c6-45f3-9d41-855cc5a36109 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:19.167588 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-85b55046-67c6-45f3-9d41-855cc5a36109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:19.170137 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e0b06a-7704-44fc-8224-04c89b4a6b71 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:19.234753 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:19.243723 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c63cf3-2238-4a13-a3b0-9519901a15c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:19.244374 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c63cf3-2238-4a13-a3b0-9519901a15c2 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:19.244683 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c63cf3-2238-4a13-a3b0-9519901a15c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:19.244957 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c63cf3-2238-4a13-a3b0-9519901a15c2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:19.273335 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c63cf3-2238-4a13-a3b0-9519901a15c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:19.279134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-30c63cf3-2238-4a13-a3b0-9519901a15c2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:49:19.279134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:19.279134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:19.279134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:19.279134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:19.279134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:19.279134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:19.279134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:19.279134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:19.279134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:19.279134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:19.279134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:19.279134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:19.279134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:19.279134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:19.279134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:19.280183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:19.280183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:19.280183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:19.280183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:19.280183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:19.280183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:19.280183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:19.280183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:19.280183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:19.280183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:19.280183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:19.280183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:19.280183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:19.280183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:19.280183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:19.280183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:19.281002 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:19.281002 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:19.281002 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:19.281002 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:19.281002 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:19.281002 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:19.281002 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:19.281002 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:19.281002 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:19.281002 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:19.281002 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:49:19.281002 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:19.283720 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 115/238] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:19.293125 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c63cf3-2238-4a13-a3b0-9519901a15c2 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:19.318524 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 124/239] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:19.352358 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b55046-67c6-45f3-9d41-855cc5a36109 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:19.437225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:19.497837 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c63cf3-2238-4a13-a3b0-9519901a15c2 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:19.558358 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea26b65b-9d53-41cd-a89e-c8b69bfe0586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:19.559201 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26b65b-9d53-41cd-a89e-c8b69bfe0586 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:19.559585 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26b65b-9d53-41cd-a89e-c8b69bfe0586 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:19.559871 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26b65b-9d53-41cd-a89e-c8b69bfe0586 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:19.560642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea26b65b-9d53-41cd-a89e-c8b69bfe0586 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:19.561253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea26b65b-9d53-41cd-a89e-c8b69bfe0586 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:19.563327 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea26b65b-9d53-41cd-a89e-c8b69bfe0586 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:19.566538 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b55046-67c6-45f3-9d41-855cc5a36109 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:19.574242 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 116/240] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:19.584007 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea26b65b-9d53-41cd-a89e-c8b69bfe0586 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:19.640598 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394e4b5-c5e6-4f33-b7b6-75a64707cc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:19.641767 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394e4b5-c5e6-4f33-b7b6-75a64707cc38 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:19.642055 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394e4b5-c5e6-4f33-b7b6-75a64707cc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:19.642372 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394e4b5-c5e6-4f33-b7b6-75a64707cc38 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:19.643898 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3394e4b5-c5e6-4f33-b7b6-75a64707cc38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:19.645421 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3394e4b5-c5e6-4f33-b7b6-75a64707cc38 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:19.647274 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 125/241] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:19.658430 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394e4b5-c5e6-4f33-b7b6-75a64707cc38 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:19.684933 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:19.738647 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea26b65b-9d53-41cd-a89e-c8b69bfe0586 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:19.788201 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9fb8f9-ea8d-4605-8ef2-95a217f26c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:19.789285 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9fb8f9-ea8d-4605-8ef2-95a217f26c82 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:19.789780 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9fb8f9-ea8d-4605-8ef2-95a217f26c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:19.790261 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9fb8f9-ea8d-4605-8ef2-95a217f26c82 None admin] PATH_INFO: `/v3/users/5de1755b051d4206a4da3e84e929994d` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:19.831036 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9fb8f9-ea8d-4605-8ef2-95a217f26c82 None admin] RBAC: Authorizing `identity:get_user(user_id=5de1755b051d4206a4da3e84e929994d)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:19.832511 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9fb8f9-ea8d-4605-8ef2-95a217f26c82 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:19.834710 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 117/242] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:19 2022] GET /identity/v3/users/5de1755b051d4206a4da3e84e929994d => generated 297 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:19.842191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9fb8f9-ea8d-4605-8ef2-95a217f26c82 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:19.897286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394e4b5-c5e6-4f33-b7b6-75a64707cc38 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:19.916033 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:19.986457 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8dd65-fff4-41e0-8a72-ca649e3394c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:19.987387 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8dd65-fff4-41e0-8a72-ca649e3394c6 None admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:19.987809 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8dd65-fff4-41e0-8a72-ca649e3394c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:19.988232 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8dd65-fff4-41e0-8a72-ca649e3394c6 None admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:19.989372 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b8dd65-fff4-41e0-8a72-ca649e3394c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:19.991617 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b8dd65-fff4-41e0-8a72-ca649e3394c6 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:20.049416 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 126/243] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Aug 5 19:49:19 2022] POST /identity/v3/users => generated 293 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:20.102732 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9fb8f9-ea8d-4605-8ef2-95a217f26c82 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:20.124539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:20.163794 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46ef32-9768-4af6-9bdc-ecf0a0e67086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:20.164432 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46ef32-9768-4af6-9bdc-ecf0a0e67086 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:20.164748 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46ef32-9768-4af6-9bdc-ecf0a0e67086 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:20.165026 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46ef32-9768-4af6-9bdc-ecf0a0e67086 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:20.275825 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e46ef32-9768-4af6-9bdc-ecf0a0e67086 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:20.277256 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e46ef32-9768-4af6-9bdc-ecf0a0e67086 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:20.339204 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47690ef9-d1a5-4e47-964b-71d3d5bbca1a None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:20.339988 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47690ef9-d1a5-4e47-964b-71d3d5bbca1a None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:20.340462 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47690ef9-d1a5-4e47-964b-71d3d5bbca1a None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:20.342461 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 127/244] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:20.348992 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54627a27-7425-4410-980f-33d619b96923 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:20.349274 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54627a27-7425-4410-980f-33d619b96923 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:20.349548 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54627a27-7425-4410-980f-33d619b96923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:20.393304 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-54627a27-7425-4410-980f-33d619b96923 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:20.395066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-54627a27-7425-4410-980f-33d619b96923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:20.432609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 118/245] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:20.439458 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46ef32-9768-4af6-9bdc-ecf0a0e67086 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:20.455071 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:20.491546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 128/246] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:20.500305 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f06439-6223-4cca-baff-c933d1f6ba96 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:20.500305 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f06439-6223-4cca-baff-c933d1f6ba96 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:20.500305 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f06439-6223-4cca-baff-c933d1f6ba96 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:20.501642 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 129/247] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:20.506562 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a91839-eb9f-4e79-ae53-491324c4df58 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:20.506856 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a91839-eb9f-4e79-ae53-491324c4df58 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:20.507134 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a91839-eb9f-4e79-ae53-491324c4df58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:20.528074 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-77a91839-eb9f-4e79-ae53-491324c4df58 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:20.529221 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-77a91839-eb9f-4e79-ae53-491324c4df58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:20.557889 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46ef32-9768-4af6-9bdc-ecf0a0e67086 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:20.617013 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb81f8aa-f2d6-4891-b1a5-1e73a1bc2560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:20.617652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81f8aa-f2d6-4891-b1a5-1e73a1bc2560 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:20.617929 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81f8aa-f2d6-4891-b1a5-1e73a1bc2560 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:20.618201 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81f8aa-f2d6-4891-b1a5-1e73a1bc2560 None admin] PATH_INFO: `/v3/projects/c0ded24d8148467b9a56fa55332feb56` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:20.619985 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb81f8aa-f2d6-4891-b1a5-1e73a1bc2560 None admin] RBAC: Authorizing `identity:get_project(project_id=c0ded24d8148467b9a56fa55332feb56)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:20.621478 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb81f8aa-f2d6-4891-b1a5-1e73a1bc2560 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:20.623346 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 119/248] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:20 2022] GET /identity/v3/projects/c0ded24d8148467b9a56fa55332feb56 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:20.624218 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb81f8aa-f2d6-4891-b1a5-1e73a1bc2560 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:20.634782 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 130/249] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:49:20.636096 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a91839-eb9f-4e79-ae53-491324c4df58 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:20.669095 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:20.747132 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:20.767232 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a91839-eb9f-4e79-ae53-491324c4df58 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:20.773021 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb81f8aa-f2d6-4891-b1a5-1e73a1bc2560 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:20.809646 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007e811-3a77-471a-998b-09b3bda0c89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:20.810193 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007e811-3a77-471a-998b-09b3bda0c89d None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:20.810477 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007e811-3a77-471a-998b-09b3bda0c89d None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:20.810810 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007e811-3a77-471a-998b-09b3bda0c89d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:20.811712 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1007e811-3a77-471a-998b-09b3bda0c89d None admin] RBAC: Adding query filter params (role.id=9de09806c19c49e7aba24d414a65eb5f, scope.project.id=c0ded24d8148467b9a56fa55332feb56, user.id=5de1755b051d4206a4da3e84e929994d) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:20.812040 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1007e811-3a77-471a-998b-09b3bda0c89d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:20.813909 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1007e811-3a77-471a-998b-09b3bda0c89d None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:20.818611 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a86386-84dc-4998-958b-d762dcbd5e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:20.819262 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a86386-84dc-4998-958b-d762dcbd5e0c None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:20.819546 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a86386-84dc-4998-958b-d762dcbd5e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:20.819820 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a86386-84dc-4998-958b-d762dcbd5e0c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:20.819988 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 131/250] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:49:20 2022] GET /identity/v3/role_assignments?user.id=5de1755b051d4206a4da3e84e929994d&scope.project.id=c0ded24d8148467b9a56fa55332feb56&role.id=9de09806c19c49e7aba24d414a65eb5f => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:20.820882 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007e811-3a77-471a-998b-09b3bda0c89d None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:20.851615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a86386-84dc-4998-958b-d762dcbd5e0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:20.859069 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-d7a86386-84dc-4998-958b-d762dcbd5e0c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:20.859069 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:20.859069 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:20.859069 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:20.859069 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:20.859069 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:20.859069 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:20.859069 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:20.859069 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:20.859069 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:20.859069 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:20.859069 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:20.859069 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:20.859069 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:20.859069 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:20.859069 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:20.860835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:20.860835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:20.860835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:20.860835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:20.860835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:20.860835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:20.860835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:20.860835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:20.860835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:20.860835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:20.860835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:20.860835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:20.860835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:20.860835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:20.860835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:20.860835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:20.861658 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:20.861658 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:20.861658 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:20.861658 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:20.861658 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:20.861658 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:20.861658 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:20.861658 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:20.861658 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:20.861658 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:20.861658 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:20.861658 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:20.863071 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 120/251] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:20.876160 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a86386-84dc-4998-958b-d762dcbd5e0c None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:21.067342 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:21.098512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007e811-3a77-471a-998b-09b3bda0c89d None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:21.189446 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e819e53-85d7-4145-95d8-b305451e2e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:21.190469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e819e53-85d7-4145-95d8-b305451e2e41 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:21.190942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e819e53-85d7-4145-95d8-b305451e2e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:21.192212 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e819e53-85d7-4145-95d8-b305451e2e41 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:21.194960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a86386-84dc-4998-958b-d762dcbd5e0c None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:21.219815 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e819e53-85d7-4145-95d8-b305451e2e41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:21.228178 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-7e819e53-85d7-4145-95d8-b305451e2e41 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:21.228178 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:21.228178 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:21.228178 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:21.228178 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:21.228178 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:21.228178 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:21.228178 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:21.228178 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:21.228178 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:21.228178 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:21.228178 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:21.228178 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:21.228178 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:21.228178 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:21.228178 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:21.229881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:21.229881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:21.229881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:21.229881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:21.229881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:21.229881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:21.229881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:21.229881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:21.229881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:21.229881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:21.229881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:21.229881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:21.229881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:21.229881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:21.229881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:21.229881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:21.231391 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:21.231391 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:21.231391 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:21.231391 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:21.231391 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:21.231391 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:21.231391 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:21.231391 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:21.231391 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:21.231391 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:21.231391 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:21.231391 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:49:21.237047 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 132/252] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Aug 5 19:49:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 416 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:21.239592 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4811a-4f2f-4a44-b0c1-05cc3db36da7 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:21.240215 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4811a-4f2f-4a44-b0c1-05cc3db36da7 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:21.240911 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4811a-4f2f-4a44-b0c1-05cc3db36da7 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:21.243110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 133/253] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:21.244304 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4811a-4f2f-4a44-b0c1-05cc3db36da7 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:21.270402 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd1dbb0-b558-467c-ae6a-185738db14db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:21.270961 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1dbb0-b558-467c-ae6a-185738db14db None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:21.271247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1dbb0-b558-467c-ae6a-185738db14db None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:21.271524 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1dbb0-b558-467c-ae6a-185738db14db None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:21.272247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd1dbb0-b558-467c-ae6a-185738db14db None admin] RBAC: Adding query filter params (name=member) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:21.272556 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd1dbb0-b558-467c-ae6a-185738db14db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:21.273989 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd1dbb0-b558-467c-ae6a-185738db14db None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:21.286783 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 121/254] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:21.288993 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c399fc-c3f9-46e3-8694-a6ee235746c1 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:21.289438 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c399fc-c3f9-46e3-8694-a6ee235746c1 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:21.289852 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c399fc-c3f9-46e3-8694-a6ee235746c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:21.312969 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:21.318953 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-36c399fc-c3f9-46e3-8694-a6ee235746c1 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:21.320651 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-36c399fc-c3f9-46e3-8694-a6ee235746c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:21.469745 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 122/255] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:21.471031 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c399fc-c3f9-46e3-8694-a6ee235746c1 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:21.565184 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4811a-4f2f-4a44-b0c1-05cc3db36da7 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:21.599277 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c399fc-c3f9-46e3-8694-a6ee235746c1 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:21.645065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:21.656919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6566c-9279-4f1f-b862-a31dd0433b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:21.657578 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6566c-9279-4f1f-b862-a31dd0433b37 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:21.657856 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6566c-9279-4f1f-b862-a31dd0433b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:21.658124 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6566c-9279-4f1f-b862-a31dd0433b37 None admin] PATH_INFO: `/v3/users/9a522bb1b075409dbbcc974adfa48476` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:21.659539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6566c-9279-4f1f-b862-a31dd0433b37 None admin] RBAC: Authorizing `identity:get_user(user_id=9a522bb1b075409dbbcc974adfa48476)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:21.661016 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6566c-9279-4f1f-b862-a31dd0433b37 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:21.662876 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 123/256] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:21 2022] GET /identity/v3/users/9a522bb1b075409dbbcc974adfa48476 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:21.663398 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6566c-9279-4f1f-b862-a31dd0433b37 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:21.669831 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc35f5-2c27-45e7-8d52-3c590ef800e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:21.670472 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc35f5-2c27-45e7-8d52-3c590ef800e9 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:21.670749 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc35f5-2c27-45e7-8d52-3c590ef800e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:21.671017 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc35f5-2c27-45e7-8d52-3c590ef800e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:21.672222 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fc35f5-2c27-45e7-8d52-3c590ef800e9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:21.672536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fc35f5-2c27-45e7-8d52-3c590ef800e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:21.673977 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fc35f5-2c27-45e7-8d52-3c590ef800e9 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:21.698715 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 134/257] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 455 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:21.699408 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc35f5-2c27-45e7-8d52-3c590ef800e9 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:21.728351 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:21.801516 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6566c-9279-4f1f-b862-a31dd0433b37 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:21.841004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf497a-5104-49c8-81aa-6eb6d9b9cf63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:21.841843 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf497a-5104-49c8-81aa-6eb6d9b9cf63 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:21.842129 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf497a-5104-49c8-81aa-6eb6d9b9cf63 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:21.842406 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf497a-5104-49c8-81aa-6eb6d9b9cf63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:21.855592 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaf497a-5104-49c8-81aa-6eb6d9b9cf63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:21.860779 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-bfaf497a-5104-49c8-81aa-6eb6d9b9cf63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:49:21.860779 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:21.860779 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:21.860779 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:21.860779 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:21.860779 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:21.860779 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:21.860779 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:21.860779 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:21.860779 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:21.860779 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:21.860779 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:21.860779 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:21.860779 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:21.860779 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:21.860779 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:21.861678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:21.861678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:21.861678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:21.861678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:21.861678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:21.861678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:21.861678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:21.861678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:21.861678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:21.861678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:21.861678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:21.861678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:21.861678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:21.861678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:21.861678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:21.861678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:21.862483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:21.862483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:21.862483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:21.862483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:21.862483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:21.862483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:21.862483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:21.862483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:21.862483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:21.862483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:21.862483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:49:21.862483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:21.863090 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 124/258] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:49:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:21.863634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf497a-5104-49c8-81aa-6eb6d9b9cf63 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:21.957081 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:21.987440 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc35f5-2c27-45e7-8d52-3c590ef800e9 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:22.010731 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf497a-5104-49c8-81aa-6eb6d9b9cf63 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:22.063694 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87caca8d-4a4a-4b4b-9042-f517f2c9b439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:22.064346 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87caca8d-4a4a-4b4b-9042-f517f2c9b439 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:22.064644 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87caca8d-4a4a-4b4b-9042-f517f2c9b439 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:22.064919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87caca8d-4a4a-4b4b-9042-f517f2c9b439 None admin] PATH_INFO: `/v3/projects/c0ded24d8148467b9a56fa55332feb56` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:22.069486 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87caca8d-4a4a-4b4b-9042-f517f2c9b439 None admin] RBAC: Authorizing `identity:get_project(project_id=c0ded24d8148467b9a56fa55332feb56)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:22.070951 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87caca8d-4a4a-4b4b-9042-f517f2c9b439 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:22.072813 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 135/259] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:21 2022] GET /identity/v3/projects/c0ded24d8148467b9a56fa55332feb56 => generated 302 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:22.073822 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87caca8d-4a4a-4b4b-9042-f517f2c9b439 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:22.086179 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28178b3-e44c-45f7-865b-1bee9cd406df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:22.086820 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28178b3-e44c-45f7-865b-1bee9cd406df None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:22.087098 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28178b3-e44c-45f7-865b-1bee9cd406df None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:22.087373 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28178b3-e44c-45f7-865b-1bee9cd406df None admin] PATH_INFO: `/v3/users/71a49e6aceb747ba9f57bea5d9f9cd9c` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:22.088804 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28178b3-e44c-45f7-865b-1bee9cd406df None admin] RBAC: Authorizing `identity:get_user(user_id=71a49e6aceb747ba9f57bea5d9f9cd9c)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:22.090239 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28178b3-e44c-45f7-865b-1bee9cd406df None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:22.096999 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 125/260] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:21 2022] GET /identity/v3/users/71a49e6aceb747ba9f57bea5d9f9cd9c => generated 283 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:22.097835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28178b3-e44c-45f7-865b-1bee9cd406df None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:22.240275 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28178b3-e44c-45f7-865b-1bee9cd406df None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:22.270667 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:22.273910 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:22.283986 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874f151-981a-4300-9fce-43b615277d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:22.284716 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874f151-981a-4300-9fce-43b615277d1d None admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:22.285010 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874f151-981a-4300-9fce-43b615277d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:22.285297 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874f151-981a-4300-9fce-43b615277d1d None admin] PATH_INFO: `/v3/projects/c0ded24d8148467b9a56fa55332feb56/users/9a522bb1b075409dbbcc974adfa48476/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:22.287594 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874f151-981a-4300-9fce-43b615277d1d None admin] RBAC: Authorizing `identity:create_grant(project_id=c0ded24d8148467b9a56fa55332feb56, user_id=9a522bb1b075409dbbcc974adfa48476, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:22.289300 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874f151-981a-4300-9fce-43b615277d1d None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:22.306216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 126/261] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Aug 5 19:49:22 2022] PUT /identity/v3/projects/c0ded24d8148467b9a56fa55332feb56/users/9a522bb1b075409dbbcc974adfa48476/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:22.307468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874f151-981a-4300-9fce-43b615277d1d None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:22.343263 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87caca8d-4a4a-4b4b-9042-f517f2c9b439 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:22.458499 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874f151-981a-4300-9fce-43b615277d1d None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:22.500500 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484dc419-1a2d-4d44-98aa-a262c80d13ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:22.501687 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dc419-1a2d-4d44-98aa-a262c80d13ac None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:22.501994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dc419-1a2d-4d44-98aa-a262c80d13ac None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:22.502279 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dc419-1a2d-4d44-98aa-a262c80d13ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:22.513358 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaeec44-b91b-4bbf-900c-7b7c3478c932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:22.514009 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaeec44-b91b-4bbf-900c-7b7c3478c932 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:22.514332 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaeec44-b91b-4bbf-900c-7b7c3478c932 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:22.514652 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaeec44-b91b-4bbf-900c-7b7c3478c932 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:22.515463 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaeec44-b91b-4bbf-900c-7b7c3478c932 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:22.515783 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaeec44-b91b-4bbf-900c-7b7c3478c932 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:22.517250 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaeec44-b91b-4bbf-900c-7b7c3478c932 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:22.546766 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 136/262] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 473 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:22.552944 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaeec44-b91b-4bbf-900c-7b7c3478c932 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:22.598770 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:22.614417 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484dc419-1a2d-4d44-98aa-a262c80d13ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:22.615907 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484dc419-1a2d-4d44-98aa-a262c80d13ac None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:22.868492 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 127/263] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:22.875317 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484dc419-1a2d-4d44-98aa-a262c80d13ac None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:22.955165 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaeec44-b91b-4bbf-900c-7b7c3478c932 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:23.117330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cfa636-63e3-4619-973b-47a9cda7887f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:23.118261 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfa636-63e3-4619-973b-47a9cda7887f None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:23.118261 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfa636-63e3-4619-973b-47a9cda7887f None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:23.118595 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfa636-63e3-4619-973b-47a9cda7887f None admin] PATH_INFO: `/v3/users/71a49e6aceb747ba9f57bea5d9f9cd9c` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:23.125959 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cfa636-63e3-4619-973b-47a9cda7887f None admin] RBAC: Authorizing `identity:get_user(user_id=71a49e6aceb747ba9f57bea5d9f9cd9c)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:23.127747 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cfa636-63e3-4619-973b-47a9cda7887f None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:23.129707 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 137/264] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:22 2022] GET /identity/v3/users/71a49e6aceb747ba9f57bea5d9f9cd9c => generated 283 bytes in 577 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:23.153848 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cfa636-63e3-4619-973b-47a9cda7887f None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:23.181769 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:23.205165 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:23.246656 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484dc419-1a2d-4d44-98aa-a262c80d13ac None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:23.377518 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc642f1-8a6d-4b6a-88e8-2984d02be813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:23.378042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc642f1-8a6d-4b6a-88e8-2984d02be813 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:23.378316 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc642f1-8a6d-4b6a-88e8-2984d02be813 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:23.378588 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc642f1-8a6d-4b6a-88e8-2984d02be813 None admin] PATH_INFO: `/v3/projects/c0ded24d8148467b9a56fa55332feb56` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:23.380609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc642f1-8a6d-4b6a-88e8-2984d02be813 None admin] RBAC: Authorizing `identity:get_project(project_id=c0ded24d8148467b9a56fa55332feb56)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:23.393397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc642f1-8a6d-4b6a-88e8-2984d02be813 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:23.395476 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 128/265] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:22 2022] GET /identity/v3/projects/c0ded24d8148467b9a56fa55332feb56 => generated 302 bytes in 520 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:23.406698 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5839e9-d75a-474e-89a5-37fdf33da994 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:23.407025 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5839e9-d75a-474e-89a5-37fdf33da994 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:23.407267 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5839e9-d75a-474e-89a5-37fdf33da994 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:23.409796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 129/266] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:23.410735 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5839e9-d75a-474e-89a5-37fdf33da994 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:23.462252 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cfa636-63e3-4619-973b-47a9cda7887f None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:23.572108 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8dffde-7982-445a-a055-6f865ce78e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:23.577255 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8dffde-7982-445a-a055-6f865ce78e73 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:23.577995 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8dffde-7982-445a-a055-6f865ce78e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:23.578969 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8dffde-7982-445a-a055-6f865ce78e73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:23.660794 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5839e9-d75a-474e-89a5-37fdf33da994 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:23.678696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8dffde-7982-445a-a055-6f865ce78e73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:23.680413 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8dffde-7982-445a-a055-6f865ce78e73 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:23.694940 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:23.702010 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b6f0d-1d07-4d21-88f7-da7a8d70d749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:23.703403 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b6f0d-1d07-4d21-88f7-da7a8d70d749 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:23.703908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b6f0d-1d07-4d21-88f7-da7a8d70d749 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:23.704354 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b6f0d-1d07-4d21-88f7-da7a8d70d749 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:23.705383 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5b6f0d-1d07-4d21-88f7-da7a8d70d749 None admin] RBAC: Adding query filter params (role.id=0eafd8a7ba2d47faaf5aecbb3360143b, scope.project.id=c0ded24d8148467b9a56fa55332feb56, user.id=71a49e6aceb747ba9f57bea5d9f9cd9c) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:23.705846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5b6f0d-1d07-4d21-88f7-da7a8d70d749 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:23.707573 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5b6f0d-1d07-4d21-88f7-da7a8d70d749 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:23.720930 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 130/267] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:49:23 2022] GET /identity/v3/role_assignments?user.id=71a49e6aceb747ba9f57bea5d9f9cd9c&scope.project.id=c0ded24d8148467b9a56fa55332feb56&role.id=0eafd8a7ba2d47faaf5aecbb3360143b => generated 608 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:23.722589 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5bab88-414a-4b60-88fe-cdba779bca5a None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:23.723100 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5bab88-414a-4b60-88fe-cdba779bca5a None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:23.723519 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5bab88-414a-4b60-88fe-cdba779bca5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:23.750940 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-fa5bab88-414a-4b60-88fe-cdba779bca5a None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:23.752465 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-fa5bab88-414a-4b60-88fe-cdba779bca5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:23.770558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:23.810172 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 138/268] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 657 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:23.816550 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199386a2-6136-41d0-a867-80010e2cba0a None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:23.817414 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199386a2-6136-41d0-a867-80010e2cba0a None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:23.817926 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199386a2-6136-41d0-a867-80010e2cba0a None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:23.819653 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 139/269] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:23.820530 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199386a2-6136-41d0-a867-80010e2cba0a None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:23.880967 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 131/270] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:23.890164 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183316c-6323-48e8-8ac3-b74fbed4a771 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:23.890978 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183316c-6323-48e8-8ac3-b74fbed4a771 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:23.891258 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183316c-6323-48e8-8ac3-b74fbed4a771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:23.920071 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-4183316c-6323-48e8-8ac3-b74fbed4a771 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:23.925408 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-4183316c-6323-48e8-8ac3-b74fbed4a771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:23.940878 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199386a2-6136-41d0-a867-80010e2cba0a None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:23.956307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:23.980837 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca651a4-3265-48aa-8d13-4b73ab82cc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:23.981845 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca651a4-3265-48aa-8d13-4b73ab82cc08 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:23.982267 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca651a4-3265-48aa-8d13-4b73ab82cc08 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:23.982677 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca651a4-3265-48aa-8d13-4b73ab82cc08 None admin] PATH_INFO: `/v3/projects/c0ded24d8148467b9a56fa55332feb56` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:23.984705 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca651a4-3265-48aa-8d13-4b73ab82cc08 None admin] RBAC: Authorizing `identity:get_project(project_id=c0ded24d8148467b9a56fa55332feb56)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:23.986824 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca651a4-3265-48aa-8d13-4b73ab82cc08 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:23.988976 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 140/271] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:23 2022] GET /identity/v3/projects/c0ded24d8148467b9a56fa55332feb56 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:23.990214 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc20a2-e5c6-469a-89e8-64eee3c0bc32 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:23.990491 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc20a2-e5c6-469a-89e8-64eee3c0bc32 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:23.990764 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc20a2-e5c6-469a-89e8-64eee3c0bc32 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:23.997166 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 141/272] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:23.998061 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc20a2-e5c6-469a-89e8-64eee3c0bc32 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:24.023580 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 132/273] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:24.025941 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4183316c-6323-48e8-8ac3-b74fbed4a771 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:24.124416 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:24.135242 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc20a2-e5c6-469a-89e8-64eee3c0bc32 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:24.148417 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4183316c-6323-48e8-8ac3-b74fbed4a771 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:24.176042 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597247f-12ea-49c8-a1b1-6f8d83197828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:24.176727 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597247f-12ea-49c8-a1b1-6f8d83197828 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:24.177012 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597247f-12ea-49c8-a1b1-6f8d83197828 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:24.177286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597247f-12ea-49c8-a1b1-6f8d83197828 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:24.185867 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504a4f6-8337-43e2-8223-c2316ea8fcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:24.186477 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504a4f6-8337-43e2-8223-c2316ea8fcd2 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:24.186756 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504a4f6-8337-43e2-8223-c2316ea8fcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:24.187044 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504a4f6-8337-43e2-8223-c2316ea8fcd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:24.188076 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a504a4f6-8337-43e2-8223-c2316ea8fcd2 None admin] RBAC: Adding query filter params (role.id=89919f784ad144529b07c8ab92ee7d48, scope.project.id=c0ded24d8148467b9a56fa55332feb56, user.id=71a49e6aceb747ba9f57bea5d9f9cd9c) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:24.188380 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a504a4f6-8337-43e2-8223-c2316ea8fcd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:24.188851 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3597247f-12ea-49c8-a1b1-6f8d83197828 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:24.189848 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a504a4f6-8337-43e2-8223-c2316ea8fcd2 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:24.193869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-3597247f-12ea-49c8-a1b1-6f8d83197828 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:24.193869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:24.193869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:24.193869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:24.193869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:24.193869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:24.193869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:24.193869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:24.193869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:24.193869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:24.193869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:24.193869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:24.193869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:24.193869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:24.193869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:24.193869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:24.194722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:24.194722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:24.194722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:24.194722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:24.194722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:24.194722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:24.194722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:24.194722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:24.194722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:24.194722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:24.194722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:24.194722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:24.194722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:24.194722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:24.194722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:24.194722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:24.195503 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:24.195503 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:24.195503 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:24.195503 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:24.195503 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:24.195503 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:24.195503 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:24.195503 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:24.195503 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:24.195503 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:24.195503 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:24.195503 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:49:24.195503 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 142/274] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:24.196168 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60dc50c-c8c9-452d-91e4-d3303b9a11f1 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:24.196869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60dc50c-c8c9-452d-91e4-d3303b9a11f1 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:24.196869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60dc50c-c8c9-452d-91e4-d3303b9a11f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:24.208952 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 133/275] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:49:24 2022] GET /identity/v3/role_assignments?user.id=71a49e6aceb747ba9f57bea5d9f9cd9c&scope.project.id=c0ded24d8148467b9a56fa55332feb56&role.id=89919f784ad144529b07c8ab92ee7d48 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:24.208952 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504a4f6-8337-43e2-8223-c2316ea8fcd2 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:24.224039 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-f60dc50c-c8c9-452d-91e4-d3303b9a11f1 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:24.225297 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-f60dc50c-c8c9-452d-91e4-d3303b9a11f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:24.267472 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:24.311460 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 143/276] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:24.312763 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60dc50c-c8c9-452d-91e4-d3303b9a11f1 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:24.354369 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504a4f6-8337-43e2-8223-c2316ea8fcd2 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:24.399301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d1d15b-4eaf-4a75-8a03-e1372be46b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:24.399928 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1d15b-4eaf-4a75-8a03-e1372be46b24 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:24.400511 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1d15b-4eaf-4a75-8a03-e1372be46b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:24.400953 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1d15b-4eaf-4a75-8a03-e1372be46b24 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:24.413437 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d1d15b-4eaf-4a75-8a03-e1372be46b24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:24.418980 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-87d1d15b-4eaf-4a75-8a03-e1372be46b24 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:49:24.418980 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:24.418980 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:24.418980 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:24.418980 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:24.418980 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:24.418980 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:24.418980 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:24.418980 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:24.418980 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:24.418980 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:24.418980 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:24.418980 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:24.418980 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:24.418980 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:24.418980 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:24.419947 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:24.419947 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:24.419947 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:24.419947 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:24.419947 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:24.419947 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:24.419947 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:24.419947 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:24.419947 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:24.419947 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:24.419947 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:24.419947 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:24.419947 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:24.419947 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:24.419947 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:24.419947 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:24.420767 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:24.420767 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:24.420767 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:24.420767 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:24.420767 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:24.420767 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:24.420767 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:24.420767 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:24.420767 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:24.420767 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:24.420767 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:49:24.420767 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:24.420767 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 134/277] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:24.422017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d1d15b-4eaf-4a75-8a03-e1372be46b24 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:24.426263 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:24.458649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60dc50c-c8c9-452d-91e4-d3303b9a11f1 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:24.501726 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3eb8e-d2bb-4db4-9464-fc96d2e675ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:24.502357 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3eb8e-d2bb-4db4-9464-fc96d2e675ed None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:24.502639 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3eb8e-d2bb-4db4-9464-fc96d2e675ed None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:24.502910 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3eb8e-d2bb-4db4-9464-fc96d2e675ed None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:24.503640 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea3eb8e-d2bb-4db4-9464-fc96d2e675ed None admin] RBAC: Adding query filter params (name=member) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:24.503946 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea3eb8e-d2bb-4db4-9464-fc96d2e675ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:24.505429 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea3eb8e-d2bb-4db4-9464-fc96d2e675ed None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:24.515954 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 144/278] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:24.517986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3eb8e-d2bb-4db4-9464-fc96d2e675ed None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:24.554624 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:24.563695 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d1d15b-4eaf-4a75-8a03-e1372be46b24 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:24.614340 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df93348-01e1-47e0-b7aa-65807d73c54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:24.614992 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df93348-01e1-47e0-b7aa-65807d73c54f None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:24.615379 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df93348-01e1-47e0-b7aa-65807d73c54f None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:24.615610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df93348-01e1-47e0-b7aa-65807d73c54f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:24.616796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df93348-01e1-47e0-b7aa-65807d73c54f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:24.618260 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df93348-01e1-47e0-b7aa-65807d73c54f None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:24.619981 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 135/279] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:24.620779 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df93348-01e1-47e0-b7aa-65807d73c54f None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:24.641723 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3eb8e-d2bb-4db4-9464-fc96d2e675ed None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:24.682627 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1d441-17cd-475e-92ac-f703be6ecc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:24.683502 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1d441-17cd-475e-92ac-f703be6ecc20 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:24.683864 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1d441-17cd-475e-92ac-f703be6ecc20 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:24.684265 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1d441-17cd-475e-92ac-f703be6ecc20 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:24.685291 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b1d441-17cd-475e-92ac-f703be6ecc20 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:24.685724 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b1d441-17cd-475e-92ac-f703be6ecc20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:24.688536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b1d441-17cd-475e-92ac-f703be6ecc20 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:24.700957 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 145/280] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:24.701886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1d441-17cd-475e-92ac-f703be6ecc20 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:24.718109 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:24.765148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df93348-01e1-47e0-b7aa-65807d73c54f None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:24.801353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cef8f5-1cc0-4dfc-a62f-5336f3f78e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:24.801996 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cef8f5-1cc0-4dfc-a62f-5336f3f78e5c None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:24.802277 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cef8f5-1cc0-4dfc-a62f-5336f3f78e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:24.802551 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cef8f5-1cc0-4dfc-a62f-5336f3f78e5c None admin] PATH_INFO: `/v3/users/379585e2a798420caa7f47df7fddadbd` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:24.819438 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cef8f5-1cc0-4dfc-a62f-5336f3f78e5c None admin] RBAC: Authorizing `identity:get_user(user_id=379585e2a798420caa7f47df7fddadbd)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:24.820916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cef8f5-1cc0-4dfc-a62f-5336f3f78e5c None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:24.822862 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 136/281] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:24 2022] GET /identity/v3/users/379585e2a798420caa7f47df7fddadbd => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:24.828975 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cef8f5-1cc0-4dfc-a62f-5336f3f78e5c None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:24.832395 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:24.855537 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1d441-17cd-475e-92ac-f703be6ecc20 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:24.899486 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b2517-1e20-4645-9079-6ec43bc3c929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:24.900664 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b2517-1e20-4645-9079-6ec43bc3c929 None admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:24.901128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b2517-1e20-4645-9079-6ec43bc3c929 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:24.901518 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b2517-1e20-4645-9079-6ec43bc3c929 None admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:24.902429 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b2517-1e20-4645-9079-6ec43bc3c929 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:24.904846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b2517-1e20-4645-9079-6ec43bc3c929 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:24.935058 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 146/282] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Aug 5 19:49:24 2022] POST /identity/v3/users => generated 293 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:24.945716 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b2517-1e20-4645-9079-6ec43bc3c929 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:25.013394 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cef8f5-1cc0-4dfc-a62f-5336f3f78e5c None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:25.059092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:25.068408 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291810ee-941b-482f-aba5-4d872602e963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:25.069253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291810ee-941b-482f-aba5-4d872602e963 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:25.070273 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291810ee-941b-482f-aba5-4d872602e963 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:25.070273 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291810ee-941b-482f-aba5-4d872602e963 None admin] PATH_INFO: `/v3/users/5de1755b051d4206a4da3e84e929994d` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:25.071443 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291810ee-941b-482f-aba5-4d872602e963 None admin] RBAC: Authorizing `identity:get_user(user_id=5de1755b051d4206a4da3e84e929994d)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:25.073062 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291810ee-941b-482f-aba5-4d872602e963 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:25.080088 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 137/283] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:24 2022] GET /identity/v3/users/5de1755b051d4206a4da3e84e929994d => generated 297 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:25.080088 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15cf09-655c-41f3-b34e-3d062d811635 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:25.080088 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15cf09-655c-41f3-b34e-3d062d811635 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:25.080088 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15cf09-655c-41f3-b34e-3d062d811635 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:25.080088 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 138/284] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:25.083648 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15cf09-655c-41f3-b34e-3d062d811635 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:25.122855 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b2517-1e20-4645-9079-6ec43bc3c929 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:25.175752 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f842c34-6649-439b-8aa7-efa537d86df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:25.181040 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f842c34-6649-439b-8aa7-efa537d86df4 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:25.181040 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f842c34-6649-439b-8aa7-efa537d86df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:25.181040 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f842c34-6649-439b-8aa7-efa537d86df4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:25.181040 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f842c34-6649-439b-8aa7-efa537d86df4 None admin] RBAC: Adding query filter params (role.id=fbd7c81b4fba4b6d9e61b1e9bb3e2046, user.id=379585e2a798420caa7f47df7fddadbd, scope.system=all) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:25.181040 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f842c34-6649-439b-8aa7-efa537d86df4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:25.181040 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f842c34-6649-439b-8aa7-efa537d86df4 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:25.186273 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 147/285] 10.222.0.39 () {56 vars in 1308 bytes} [Fri Aug 5 19:49:24 2022] GET /identity/v3/role_assignments?user.id=379585e2a798420caa7f47df7fddadbd&scope.system=all&role.id=fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 226 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:25.187509 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83141c-67cc-4dfd-988c-01d6b12236c1 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:25.187792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83141c-67cc-4dfd-988c-01d6b12236c1 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:25.188064 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83141c-67cc-4dfd-988c-01d6b12236c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:25.219242 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-aa83141c-67cc-4dfd-988c-01d6b12236c1 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:25.220622 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-aa83141c-67cc-4dfd-988c-01d6b12236c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:25.241490 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:25.250589 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15cf09-655c-41f3-b34e-3d062d811635 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:25.309587 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e329d5c-7a19-46fb-a95d-9d32195fac1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:25.310618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e329d5c-7a19-46fb-a95d-9d32195fac1e None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:25.311114 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e329d5c-7a19-46fb-a95d-9d32195fac1e None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:25.311591 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e329d5c-7a19-46fb-a95d-9d32195fac1e None admin] PATH_INFO: `/v3/projects/c0ded24d8148467b9a56fa55332feb56` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:25.313567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e329d5c-7a19-46fb-a95d-9d32195fac1e None admin] RBAC: Authorizing `identity:get_project(project_id=c0ded24d8148467b9a56fa55332feb56)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:25.315436 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e329d5c-7a19-46fb-a95d-9d32195fac1e None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:25.319258 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 139/286] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:25 2022] GET /identity/v3/projects/c0ded24d8148467b9a56fa55332feb56 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:25.325477 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e329d5c-7a19-46fb-a95d-9d32195fac1e None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:25.377560 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 148/287] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:25.403896 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83141c-67cc-4dfd-988c-01d6b12236c1 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:25.414375 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:25.475542 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e329d5c-7a19-46fb-a95d-9d32195fac1e None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:25.536952 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83141c-67cc-4dfd-988c-01d6b12236c1 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:25.537825 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d6762-cc66-485b-a3b3-83da93fab193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:25.539260 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d6762-cc66-485b-a3b3-83da93fab193 None admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:25.540372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d6762-cc66-485b-a3b3-83da93fab193 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:25.541164 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d6762-cc66-485b-a3b3-83da93fab193 None admin] PATH_INFO: `/v3/projects/c0ded24d8148467b9a56fa55332feb56/users/5de1755b051d4206a4da3e84e929994d/roles/9de09806c19c49e7aba24d414a65eb5f` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:25.544621 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d6762-cc66-485b-a3b3-83da93fab193 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0ded24d8148467b9a56fa55332feb56, user_id=5de1755b051d4206a4da3e84e929994d, role_id=9de09806c19c49e7aba24d414a65eb5f)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:25.547690 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d6762-cc66-485b-a3b3-83da93fab193 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:25.575469 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 140/288] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Aug 5 19:49:25 2022] PUT /identity/v3/projects/c0ded24d8148467b9a56fa55332feb56/users/5de1755b051d4206a4da3e84e929994d/roles/9de09806c19c49e7aba24d414a65eb5f => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:49:25.595947 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b192ea1-e6cf-467d-b8f1-52e390c945c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:25.597094 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b192ea1-e6cf-467d-b8f1-52e390c945c3 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:25.597628 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b192ea1-e6cf-467d-b8f1-52e390c945c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:25.598162 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b192ea1-e6cf-467d-b8f1-52e390c945c3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:25.622620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b192ea1-e6cf-467d-b8f1-52e390c945c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:25.629619 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-0b192ea1-e6cf-467d-b8f1-52e390c945c3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:25.629619 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:25.629619 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:25.629619 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:25.629619 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:25.629619 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:25.629619 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:25.629619 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:25.629619 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:25.629619 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:25.629619 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:25.629619 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:25.629619 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:25.629619 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:25.629619 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:25.629619 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:25.630611 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:25.630611 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:25.630611 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:25.630611 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:25.630611 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:25.630611 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:25.630611 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:25.630611 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:25.630611 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:25.630611 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:25.630611 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:25.630611 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:25.630611 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:25.630611 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:25.630611 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:25.630611 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:25.631388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:25.631388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:25.631388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:25.631388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:25.631388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:25.631388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:25.631388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:25.631388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:25.631388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:25.631388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:25.631388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:25.631388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:49:25.633845 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 149/289] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:25.639976 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d6762-cc66-485b-a3b3-83da93fab193 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:25.793065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d6762-cc66-485b-a3b3-83da93fab193 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:25.823659 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:25.835439 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6e04b-011c-4ebd-89ef-203d225aceb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:25.836075 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6e04b-011c-4ebd-89ef-203d225aceb4 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:25.836356 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6e04b-011c-4ebd-89ef-203d225aceb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:25.836664 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6e04b-011c-4ebd-89ef-203d225aceb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:25.837398 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce6e04b-011c-4ebd-89ef-203d225aceb4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:25.837772 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce6e04b-011c-4ebd-89ef-203d225aceb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:25.839359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce6e04b-011c-4ebd-89ef-203d225aceb4 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:25.850156 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 141/290] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:25.858022 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b192ea1-e6cf-467d-b8f1-52e390c945c3 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:25.863723 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:25.976905 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b192ea1-e6cf-467d-b8f1-52e390c945c3 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:26.029966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1381fe-ad0f-47e7-9041-ef51eaecf55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:26.030698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1381fe-ad0f-47e7-9041-ef51eaecf55d None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:26.031308 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1381fe-ad0f-47e7-9041-ef51eaecf55d None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:26.031586 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1381fe-ad0f-47e7-9041-ef51eaecf55d None admin] PATH_INFO: `/v3/users/9a522bb1b075409dbbcc974adfa48476` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:26.033359 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1381fe-ad0f-47e7-9041-ef51eaecf55d None admin] RBAC: Authorizing `identity:get_user(user_id=9a522bb1b075409dbbcc974adfa48476)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:26.034845 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1381fe-ad0f-47e7-9041-ef51eaecf55d None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:26.036601 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 150/291] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:25 2022] GET /identity/v3/users/9a522bb1b075409dbbcc974adfa48476 => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:26.042444 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6e04b-011c-4ebd-89ef-203d225aceb4 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:26.205758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6e04b-011c-4ebd-89ef-203d225aceb4 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:26.259166 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df5a0a-0d7f-4fa9-a9e6-40d7913a3bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:26.260238 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df5a0a-0d7f-4fa9-a9e6-40d7913a3bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:26.260783 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df5a0a-0d7f-4fa9-a9e6-40d7913a3bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:26.261289 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df5a0a-0d7f-4fa9-a9e6-40d7913a3bc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:26.384531 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:26.418740 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04df5a0a-0d7f-4fa9-a9e6-40d7913a3bc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:26.421303 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04df5a0a-0d7f-4fa9-a9e6-40d7913a3bc5 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:26.652705 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 142/292] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 19:49:26.659963 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1381fe-ad0f-47e7-9041-ef51eaecf55d None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:26.788907 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:26.844336 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1381fe-ad0f-47e7-9041-ef51eaecf55d None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:26.902259 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c036f0-0dd1-42e8-9dbf-3459c912ba67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:26.902910 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c036f0-0dd1-42e8-9dbf-3459c912ba67 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:26.903193 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c036f0-0dd1-42e8-9dbf-3459c912ba67 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:26.903469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c036f0-0dd1-42e8-9dbf-3459c912ba67 None admin] PATH_INFO: `/v3/projects/c0ded24d8148467b9a56fa55332feb56` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:26.907756 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c036f0-0dd1-42e8-9dbf-3459c912ba67 None admin] RBAC: Authorizing `identity:get_project(project_id=c0ded24d8148467b9a56fa55332feb56)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:26.909312 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c036f0-0dd1-42e8-9dbf-3459c912ba67 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:26.911436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 151/293] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:26 2022] GET /identity/v3/projects/c0ded24d8148467b9a56fa55332feb56 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:26.917250 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df5a0a-0d7f-4fa9-a9e6-40d7913a3bc5 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:27.059804 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:27.076915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df5a0a-0d7f-4fa9-a9e6-40d7913a3bc5 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:27.127738 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada6e00c-1b6f-4ce3-9920-6b59e49768eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:27.128366 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada6e00c-1b6f-4ce3-9920-6b59e49768eb None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:27.128669 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada6e00c-1b6f-4ce3-9920-6b59e49768eb None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:27.128949 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada6e00c-1b6f-4ce3-9920-6b59e49768eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:27.129754 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada6e00c-1b6f-4ce3-9920-6b59e49768eb None admin] RBAC: Adding query filter params (role.id=fbd7c81b4fba4b6d9e61b1e9bb3e2046, scope.project.id=c0ded24d8148467b9a56fa55332feb56, user.id=9a522bb1b075409dbbcc974adfa48476) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:27.130068 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada6e00c-1b6f-4ce3-9920-6b59e49768eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:27.131517 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada6e00c-1b6f-4ce3-9920-6b59e49768eb None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:27.142182 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 143/294] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:49:26 2022] GET /identity/v3/role_assignments?user.id=9a522bb1b075409dbbcc974adfa48476&scope.project.id=c0ded24d8148467b9a56fa55332feb56&role.id=fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:27.891017 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a51d6-8426-4f30-a1dd-afbcf3ee8be1 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:27.891604 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a51d6-8426-4f30-a1dd-afbcf3ee8be1 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:27.891604 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a51d6-8426-4f30-a1dd-afbcf3ee8be1 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:27.893224 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 152/295] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:27.898266 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045d824-e28c-44c8-a6bc-2ae1ea67e80f None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:27.898720 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045d824-e28c-44c8-a6bc-2ae1ea67e80f None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:27.899011 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045d824-e28c-44c8-a6bc-2ae1ea67e80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:27.918404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-6045d824-e28c-44c8-a6bc-2ae1ea67e80f None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:27.919625 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-6045d824-e28c-44c8-a6bc-2ae1ea67e80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:27.992282 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 144/296] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:28.018559 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7a51d6-8426-4f30-a1dd-afbcf3ee8be1 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:28.020787 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217232e-4f33-4709-a649-7075f271e6ea None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:28.021254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217232e-4f33-4709-a649-7075f271e6ea None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:28.021715 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217232e-4f33-4709-a649-7075f271e6ea None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:28.024548 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 145/297] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:28.032465 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b62d34-86c3-4006-9d13-93cd2d8deb6c None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:28.032805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b62d34-86c3-4006-9d13-93cd2d8deb6c None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:28.033100 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b62d34-86c3-4006-9d13-93cd2d8deb6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:28.056777 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-34b62d34-86c3-4006-9d13-93cd2d8deb6c None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:28.058204 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-34b62d34-86c3-4006-9d13-93cd2d8deb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:28.160619 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 146/298] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:49:28.161649 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23d233-952c-4401-9238-480260bfd37e None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:28.161976 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23d233-952c-4401-9238-480260bfd37e None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:28.162319 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23d233-952c-4401-9238-480260bfd37e None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:28.163720 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 147/299] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:28.168757 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa5815-a0e9-4621-92f3-758e8de1d357 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:28.169046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa5815-a0e9-4621-92f3-758e8de1d357 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:28.169319 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa5815-a0e9-4621-92f3-758e8de1d357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:28.180970 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7a51d6-8426-4f30-a1dd-afbcf3ee8be1 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:28.192175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-84fa5815-a0e9-4621-92f3-758e8de1d357 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:28.193901 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-84fa5815-a0e9-4621-92f3-758e8de1d357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:28.206298 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:28.229700 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae186229-5dfd-4a86-91e5-72df8a898690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:28.230818 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae186229-5dfd-4a86-91e5-72df8a898690 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:28.231109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae186229-5dfd-4a86-91e5-72df8a898690 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:28.231392 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae186229-5dfd-4a86-91e5-72df8a898690 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:28.245275 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:28.249330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae186229-5dfd-4a86-91e5-72df8a898690 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:28.261816 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-ae186229-5dfd-4a86-91e5-72df8a898690 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:28.261816 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:28.261816 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:28.261816 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:28.261816 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:28.261816 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:28.261816 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:28.261816 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:28.261816 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:28.261816 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:28.261816 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:28.261816 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:28.261816 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:28.261816 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:28.261816 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:28.261816 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:28.263563 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:28.263563 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:28.263563 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:28.263563 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:28.263563 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:28.263563 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:28.263563 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:28.263563 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:28.263563 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:28.263563 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:28.263563 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:28.263563 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:28.263563 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:28.263563 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:28.263563 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:28.263563 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:28.265827 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:28.265827 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:28.265827 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:28.265827 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:28.265827 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:28.265827 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:28.265827 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:28.265827 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:28.265827 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:28.265827 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:28.265827 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:28.265827 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:49:28.265827 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 153/300] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:28.267115 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae186229-5dfd-4a86-91e5-72df8a898690 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:28.289560 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 148/301] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:49:28.298335 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa5815-a0e9-4621-92f3-758e8de1d357 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:28.415122 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae186229-5dfd-4a86-91e5-72df8a898690 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:28.451965 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa5815-a0e9-4621-92f3-758e8de1d357 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:28.465951 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee492673-3569-4c37-9da6-92fe853ff506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:28.466595 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee492673-3569-4c37-9da6-92fe853ff506 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:28.466865 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee492673-3569-4c37-9da6-92fe853ff506 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:28.467193 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee492673-3569-4c37-9da6-92fe853ff506 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:28.478142 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee492673-3569-4c37-9da6-92fe853ff506 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:28.483132 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-ee492673-3569-4c37-9da6-92fe853ff506 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:49:28.483132 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:28.483132 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:28.483132 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:28.483132 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:28.483132 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:28.483132 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:28.483132 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:28.483132 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:28.483132 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:28.483132 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:28.483132 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:28.483132 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:28.483132 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:28.483132 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:28.483132 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:28.485213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:28.485213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:28.485213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:28.485213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:28.485213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:28.485213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:28.485213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:28.485213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:28.485213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:28.485213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:28.485213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:28.485213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:28.485213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:28.485213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:28.485213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:28.485213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:28.486041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:28.486041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:28.486041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:28.486041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:28.486041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:28.486041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:28.486041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:28.486041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:28.486041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:28.486041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:28.486041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:49:28.486041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:49:28.486041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 154/302] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:28.486041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee492673-3569-4c37-9da6-92fe853ff506 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:28.499121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522b69b-e37c-4d81-a333-c25b369e2fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:28.499859 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522b69b-e37c-4d81-a333-c25b369e2fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:28.500132 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522b69b-e37c-4d81-a333-c25b369e2fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:28.500404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522b69b-e37c-4d81-a333-c25b369e2fc1 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:28.501157 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522b69b-e37c-4d81-a333-c25b369e2fc1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:28.501456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522b69b-e37c-4d81-a333-c25b369e2fc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:28.502926 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522b69b-e37c-4d81-a333-c25b369e2fc1 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:28.511913 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 149/303] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:28.513335 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435263b-0488-4b8b-a47c-e425f68d507b None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:28.513667 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435263b-0488-4b8b-a47c-e425f68d507b None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:28.513946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435263b-0488-4b8b-a47c-e425f68d507b None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:28.515452 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 150/304] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:28.516369 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435263b-0488-4b8b-a47c-e425f68d507b None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:28.520991 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:28.565775 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:28.618173 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee492673-3569-4c37-9da6-92fe853ff506 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:28.641360 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435263b-0488-4b8b-a47c-e425f68d507b None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:28.658838 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f184a-bd63-4e77-97f0-1f2ea747f0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:28.659487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f184a-bd63-4e77-97f0-1f2ea747f0a5 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:28.659821 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f184a-bd63-4e77-97f0-1f2ea747f0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:28.660029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f184a-bd63-4e77-97f0-1f2ea747f0a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:28.661196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286f184a-bd63-4e77-97f0-1f2ea747f0a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:28.662646 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286f184a-bd63-4e77-97f0-1f2ea747f0a5 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:28.664234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 155/305] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:28.665113 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f184a-bd63-4e77-97f0-1f2ea747f0a5 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:28.684830 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0303d6-a2a2-46df-a845-b990d9fb3ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:28.685365 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0303d6-a2a2-46df-a845-b990d9fb3ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:28.685641 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0303d6-a2a2-46df-a845-b990d9fb3ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:28.685916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0303d6-a2a2-46df-a845-b990d9fb3ef7 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:28.686627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0303d6-a2a2-46df-a845-b990d9fb3ef7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:28.686963 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0303d6-a2a2-46df-a845-b990d9fb3ef7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:28.688559 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0303d6-a2a2-46df-a845-b990d9fb3ef7 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:28.697644 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 151/306] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:28.698824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9dd65a-89c7-495d-938f-a0ce6d432fdf None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:28.699105 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9dd65a-89c7-495d-938f-a0ce6d432fdf None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:28.699376 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9dd65a-89c7-495d-938f-a0ce6d432fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:28.719423 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-fb9dd65a-89c7-495d-938f-a0ce6d432fdf None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:28.721217 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-fb9dd65a-89c7-495d-938f-a0ce6d432fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:28.787352 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:28.792339 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f184a-bd63-4e77-97f0-1f2ea747f0a5 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:28.799899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 152/307] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:28.800754 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9dd65a-89c7-495d-938f-a0ce6d432fdf None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:28.834612 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:28.844285 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864b41b-82d2-4eec-8a3b-98def655e4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:28.844972 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864b41b-82d2-4eec-8a3b-98def655e4c2 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:28.845794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864b41b-82d2-4eec-8a3b-98def655e4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:28.845794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864b41b-82d2-4eec-8a3b-98def655e4c2 None admin] PATH_INFO: `/v3/users/379585e2a798420caa7f47df7fddadbd` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:28.847882 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d864b41b-82d2-4eec-8a3b-98def655e4c2 None admin] RBAC: Authorizing `identity:get_user(user_id=379585e2a798420caa7f47df7fddadbd)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:28.849248 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d864b41b-82d2-4eec-8a3b-98def655e4c2 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:28.850872 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 156/308] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:28 2022] GET /identity/v3/users/379585e2a798420caa7f47df7fddadbd => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:28.851644 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864b41b-82d2-4eec-8a3b-98def655e4c2 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:28.916117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9dd65a-89c7-495d-938f-a0ce6d432fdf None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:28.970741 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f8610d-259d-4731-a963-6ce1b189e95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:28.971703 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8610d-259d-4731-a963-6ce1b189e95b None admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:28.972172 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8610d-259d-4731-a963-6ce1b189e95b None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:28.972622 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8610d-259d-4731-a963-6ce1b189e95b None admin] PATH_INFO: `/v3/groups` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:28.973832 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f8610d-259d-4731-a963-6ce1b189e95b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:28.976476 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f8610d-259d-4731-a963-6ce1b189e95b None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:28.987147 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864b41b-82d2-4eec-8a3b-98def655e4c2 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:28.996049 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 153/309] 10.222.0.39 () {60 vars in 1134 bytes} [Fri Aug 5 19:49:28 2022] POST /identity/v3/groups => generated 229 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:28.996904 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f8610d-259d-4731-a963-6ce1b189e95b None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:29.047865 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f763cc7-62ee-43a5-940d-8a2d5ab8ee5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:29.049060 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f763cc7-62ee-43a5-940d-8a2d5ab8ee5c None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:29.049446 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f763cc7-62ee-43a5-940d-8a2d5ab8ee5c None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:29.049881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f763cc7-62ee-43a5-940d-8a2d5ab8ee5c None admin] PATH_INFO: `/v3/users/2a793618c68a49fb84c23d606d32fed1` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:29.071041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f763cc7-62ee-43a5-940d-8a2d5ab8ee5c None admin] RBAC: Authorizing `identity:get_user(user_id=2a793618c68a49fb84c23d606d32fed1)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:29.072763 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f763cc7-62ee-43a5-940d-8a2d5ab8ee5c None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:29.080872 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 157/310] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:28 2022] GET /identity/v3/users/2a793618c68a49fb84c23d606d32fed1 => generated 293 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:29.080872 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f763cc7-62ee-43a5-940d-8a2d5ab8ee5c None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:29.082988 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:29.161704 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f8610d-259d-4731-a963-6ce1b189e95b None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:29.175824 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:29.208212 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc889d1e-fa83-4475-b43b-036ef1975942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:29.209257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc889d1e-fa83-4475-b43b-036ef1975942 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:29.209807 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc889d1e-fa83-4475-b43b-036ef1975942 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:29.210438 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc889d1e-fa83-4475-b43b-036ef1975942 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:29.234475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc889d1e-fa83-4475-b43b-036ef1975942 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:29.234475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-cc889d1e-fa83-4475-b43b-036ef1975942 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:49:29.234475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:29.234475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:29.234475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:29.234475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:29.234475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:29.234475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:29.234475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:29.234475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:29.234475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:29.234475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:29.234475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:29.234475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:29.236019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:29.236019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:29.236019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:29.236019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:29.236019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:29.236019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:29.236019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:29.236019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:29.236019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:29.236019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:29.236019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:29.236019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:29.236019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:29.236019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:29.236019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:29.236019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:29.236019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:29.237785 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:29.237785 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:29.237785 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:29.237785 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:29.237785 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:29.237785 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:29.237785 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:29.237785 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:29.237785 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:29.237785 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:29.237785 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:29.237785 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:29.237785 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:49:29.237785 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:29.237785 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 154/311] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:29.239225 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc889d1e-fa83-4475-b43b-036ef1975942 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:29.247539 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f763cc7-62ee-43a5-940d-8a2d5ab8ee5c None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:29.293107 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635d360-8e89-40d4-afef-0766222dce1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:29.294465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635d360-8e89-40d4-afef-0766222dce1f None admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:29.294739 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635d360-8e89-40d4-afef-0766222dce1f None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:29.295840 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635d360-8e89-40d4-afef-0766222dce1f None admin] PATH_INFO: `/v3/system/users/379585e2a798420caa7f47df7fddadbd/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:29.298145 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635d360-8e89-40d4-afef-0766222dce1f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=379585e2a798420caa7f47df7fddadbd, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:29.300002 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635d360-8e89-40d4-afef-0766222dce1f None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:29.312392 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 158/312] 10.222.0.39 () {58 vars in 1334 bytes} [Fri Aug 5 19:49:29 2022] PUT /identity/v3/system/users/379585e2a798420caa7f47df7fddadbd/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:29.318539 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635d360-8e89-40d4-afef-0766222dce1f None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:29.396360 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc889d1e-fa83-4475-b43b-036ef1975942 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:29.431003 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:29.446309 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635d360-8e89-40d4-afef-0766222dce1f None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:29.465970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a519d-c700-42a2-ab08-6559103ffd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:29.467159 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:29.467514 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a519d-c700-42a2-ab08-6559103ffd68 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:29.469194 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a519d-c700-42a2-ab08-6559103ffd68 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:29.469943 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a519d-c700-42a2-ab08-6559103ffd68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:29.470975 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a519d-c700-42a2-ab08-6559103ffd68 None admin] RBAC: Adding query filter params (role.id=9de09806c19c49e7aba24d414a65eb5f, user.id=2a793618c68a49fb84c23d606d32fed1, scope.system=all) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:29.471527 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a519d-c700-42a2-ab08-6559103ffd68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:29.477663 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a519d-c700-42a2-ab08-6559103ffd68 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:29.488491 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 155/313] 10.222.0.39 () {56 vars in 1308 bytes} [Fri Aug 5 19:49:29 2022] GET /identity/v3/role_assignments?user.id=2a793618c68a49fb84c23d606d32fed1&scope.system=all&role.id=9de09806c19c49e7aba24d414a65eb5f => generated 226 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:29.518605 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e43323-def4-40aa-b421-8d37e5ae85e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:29.519639 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e43323-def4-40aa-b421-8d37e5ae85e5 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:29.520119 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e43323-def4-40aa-b421-8d37e5ae85e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:29.520628 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e43323-def4-40aa-b421-8d37e5ae85e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:29.521901 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e43323-def4-40aa-b421-8d37e5ae85e5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:29.522423 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e43323-def4-40aa-b421-8d37e5ae85e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:29.525426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e43323-def4-40aa-b421-8d37e5ae85e5 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:29.540961 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 159/314] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:29.548750 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a519d-c700-42a2-ab08-6559103ffd68 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:29.692732 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a519d-c700-42a2-ab08-6559103ffd68 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:29.733079 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2246d1a-1b72-4d25-a18a-b90159fb8730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:29.733773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2246d1a-1b72-4d25-a18a-b90159fb8730 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:29.734711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2246d1a-1b72-4d25-a18a-b90159fb8730 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:29.734995 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2246d1a-1b72-4d25-a18a-b90159fb8730 None admin] PATH_INFO: `/v3/users/5de1755b051d4206a4da3e84e929994d` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:29.736227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2246d1a-1b72-4d25-a18a-b90159fb8730 None admin] RBAC: Authorizing `identity:get_user(user_id=5de1755b051d4206a4da3e84e929994d)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:29.737826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2246d1a-1b72-4d25-a18a-b90159fb8730 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:29.740143 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 156/315] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:29 2022] GET /identity/v3/users/5de1755b051d4206a4da3e84e929994d => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:29.745481 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e43323-def4-40aa-b421-8d37e5ae85e5 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:29.861934 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e43323-def4-40aa-b421-8d37e5ae85e5 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:29.903045 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7cd21-d997-4f63-a8b9-d46bde0d308d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:29.904096 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7cd21-d997-4f63-a8b9-d46bde0d308d None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:29.904596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7cd21-d997-4f63-a8b9-d46bde0d308d None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:29.905088 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7cd21-d997-4f63-a8b9-d46bde0d308d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:29.978801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:30.001889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc7cd21-d997-4f63-a8b9-d46bde0d308d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:30.003284 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc7cd21-d997-4f63-a8b9-d46bde0d308d None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:30.111820 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 160/316] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:30.122634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2246d1a-1b72-4d25-a18a-b90159fb8730 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:30.190721 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:30.254631 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2246d1a-1b72-4d25-a18a-b90159fb8730 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:30.309650 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948831a-3a3a-444a-8edd-7d85f32c2c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:30.310736 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948831a-3a3a-444a-8edd-7d85f32c2c37 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:30.311040 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948831a-3a3a-444a-8edd-7d85f32c2c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:30.311507 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948831a-3a3a-444a-8edd-7d85f32c2c37 None admin] PATH_INFO: `/v3/projects/c0ded24d8148467b9a56fa55332feb56` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:30.314215 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7948831a-3a3a-444a-8edd-7d85f32c2c37 None admin] RBAC: Authorizing `identity:get_project(project_id=c0ded24d8148467b9a56fa55332feb56)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:30.317029 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7948831a-3a3a-444a-8edd-7d85f32c2c37 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:30.320769 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 157/317] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:30 2022] GET /identity/v3/projects/c0ded24d8148467b9a56fa55332feb56 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:30.326925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7cd21-d997-4f63-a8b9-d46bde0d308d None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:30.455735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:30.489806 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7cd21-d997-4f63-a8b9-d46bde0d308d None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:30.549001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd0042a-d5db-4683-80ca-4c39ffc8eeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:30.550450 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0042a-d5db-4683-80ca-4c39ffc8eeee None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:30.551133 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0042a-d5db-4683-80ca-4c39ffc8eeee None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:30.551893 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0042a-d5db-4683-80ca-4c39ffc8eeee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:30.553662 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd0042a-d5db-4683-80ca-4c39ffc8eeee None admin] RBAC: Adding query filter params (role.id=9de09806c19c49e7aba24d414a65eb5f, scope.project.id=c0ded24d8148467b9a56fa55332feb56, user.id=5de1755b051d4206a4da3e84e929994d) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:30.554508 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd0042a-d5db-4683-80ca-4c39ffc8eeee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:30.557405 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd0042a-d5db-4683-80ca-4c39ffc8eeee None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:30.569680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 161/318] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:49:30 2022] GET /identity/v3/role_assignments?user.id=5de1755b051d4206a4da3e84e929994d&scope.project.id=c0ded24d8148467b9a56fa55332feb56&role.id=9de09806c19c49e7aba24d414a65eb5f => generated 608 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:31.336799 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1574991-f137-444c-89b0-472b0aa03e58 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:31.337676 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1574991-f137-444c-89b0-472b0aa03e58 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:31.337676 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1574991-f137-444c-89b0-472b0aa03e58 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:31.339193 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 158/319] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:49:31.344736 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5f83f-0d8a-4339-8785-ed50682f5e6f None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:31.345494 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5f83f-0d8a-4339-8785-ed50682f5e6f None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:31.345494 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5f83f-0d8a-4339-8785-ed50682f5e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:31.365762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-f7b5f83f-0d8a-4339-8785-ed50682f5e6f None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:31.366946 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-f7b5f83f-0d8a-4339-8785-ed50682f5e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:31.452720 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 162/320] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:31.476637 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1574991-f137-444c-89b0-472b0aa03e58 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:31.568995 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef37bf9d-590c-4c15-b0bc-9168c0fd4ced None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:31.569287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef37bf9d-590c-4c15-b0bc-9168c0fd4ced None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:31.569564 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef37bf9d-590c-4c15-b0bc-9168c0fd4ced None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:31.570925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 163/321] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:31.576072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318479a-0b5e-483d-a184-0f6f1c84cd47 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:31.576471 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318479a-0b5e-483d-a184-0f6f1c84cd47 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:31.577306 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318479a-0b5e-483d-a184-0f6f1c84cd47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:31.597937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-a318479a-0b5e-483d-a184-0f6f1c84cd47 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:31.599308 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-a318479a-0b5e-483d-a184-0f6f1c84cd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:31.617058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1574991-f137-444c-89b0-472b0aa03e58 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:31.641067 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:31.665680 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97c44b-0ccc-4bf4-a2ce-7264992a3f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:31.666342 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97c44b-0ccc-4bf4-a2ce-7264992a3f30 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:31.666618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97c44b-0ccc-4bf4-a2ce-7264992a3f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:31.666895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97c44b-0ccc-4bf4-a2ce-7264992a3f30 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:31.668050 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d97c44b-0ccc-4bf4-a2ce-7264992a3f30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:31.669607 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d97c44b-0ccc-4bf4-a2ce-7264992a3f30 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:31.679109 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 159/322] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:31.681014 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97c44b-0ccc-4bf4-a2ce-7264992a3f30 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:31.695471 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 164/323] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:31.718956 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318479a-0b5e-483d-a184-0f6f1c84cd47 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:31.722340 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:31.801758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97c44b-0ccc-4bf4-a2ce-7264992a3f30 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:31.835976 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318479a-0b5e-483d-a184-0f6f1c84cd47 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:31.837481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41486a69-964b-4e3b-9d56-9893c889adba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:31.838098 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41486a69-964b-4e3b-9d56-9893c889adba None admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:31.838379 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41486a69-964b-4e3b-9d56-9893c889adba None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:31.838657 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41486a69-964b-4e3b-9d56-9893c889adba None admin] PATH_INFO: `/v3/groups` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:31.839363 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41486a69-964b-4e3b-9d56-9893c889adba None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:31.840909 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41486a69-964b-4e3b-9d56-9893c889adba None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:31.857710 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 160/324] 10.222.0.39 () {60 vars in 1134 bytes} [Fri Aug 5 19:49:31 2022] POST /identity/v3/groups => generated 226 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:49:31.860738 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7c3aa-0d70-46d9-b7b6-ffd7acba14ad None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:31.861227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7c3aa-0d70-46d9-b7b6-ffd7acba14ad None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:31.861682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7c3aa-0d70-46d9-b7b6-ffd7acba14ad None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:31.864002 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 161/325] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:31.871795 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3012881-0ad7-43e7-bf55-9de5cbfcd44d None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:31.872063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3012881-0ad7-43e7-bf55-9de5cbfcd44d None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:31.872338 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3012881-0ad7-43e7-bf55-9de5cbfcd44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:31.878737 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c423ae-e167-477e-b548-7f13f54596cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:31.880180 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c423ae-e167-477e-b548-7f13f54596cd None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:31.880896 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c423ae-e167-477e-b548-7f13f54596cd None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:31.881315 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c423ae-e167-477e-b548-7f13f54596cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:31.892960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-a3012881-0ad7-43e7-bf55-9de5cbfcd44d None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:31.894186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-a3012881-0ad7-43e7-bf55-9de5cbfcd44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:31.899358 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c423ae-e167-477e-b548-7f13f54596cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:31.909153 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-65c423ae-e167-477e-b548-7f13f54596cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:31.909153 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:31.909153 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:31.909153 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:31.909153 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:31.909153 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:31.909153 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:31.909153 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:31.909153 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:31.909153 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:31.909153 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:31.909153 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:31.909153 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:31.909153 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:31.909153 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:31.909153 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:31.910801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:31.910801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:31.910801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:31.910801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:31.910801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:31.910801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:31.910801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:31.910801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:31.910801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:31.910801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:31.910801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:31.910801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:31.910801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:31.910801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:31.910801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:31.910801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:31.912247 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:31.912247 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:31.912247 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:31.912247 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:31.912247 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:31.912247 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:31.912247 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:31.912247 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:31.912247 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:31.912247 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:31.912247 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:31.912247 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:49:31.915097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 165/326] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:31.925236 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c423ae-e167-477e-b548-7f13f54596cd None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:31.961628 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:32.012897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 162/327] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:49:32.028941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:32.035808 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3012881-0ad7-43e7-bf55-9de5cbfcd44d None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:32.081183 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c423ae-e167-477e-b548-7f13f54596cd None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:32.138256 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61e0da-18fe-41e5-afc5-1f12aea09612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:32.141894 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61e0da-18fe-41e5-afc5-1f12aea09612 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.141894 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61e0da-18fe-41e5-afc5-1f12aea09612 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.141894 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61e0da-18fe-41e5-afc5-1f12aea09612 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.144094 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d61e0da-18fe-41e5-afc5-1f12aea09612 None admin] RBAC: Adding query filter params (name=member) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:32.144094 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d61e0da-18fe-41e5-afc5-1f12aea09612 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:32.145700 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d61e0da-18fe-41e5-afc5-1f12aea09612 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:32.159939 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 166/328] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:32.166018 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61e0da-18fe-41e5-afc5-1f12aea09612 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:32.199975 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3012881-0ad7-43e7-bf55-9de5cbfcd44d None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:32.252154 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73444dc4-84cb-4b69-ab38-0e4e51f6570c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:32.253308 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73444dc4-84cb-4b69-ab38-0e4e51f6570c None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.253734 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73444dc4-84cb-4b69-ab38-0e4e51f6570c None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.254138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73444dc4-84cb-4b69-ab38-0e4e51f6570c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.271641 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73444dc4-84cb-4b69-ab38-0e4e51f6570c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:32.276943 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-73444dc4-84cb-4b69-ab38-0e4e51f6570c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:49:32.276943 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:32.276943 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:32.276943 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:32.276943 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:32.276943 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:32.276943 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:32.276943 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:32.276943 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:32.276943 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:32.276943 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:32.276943 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:32.276943 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:32.276943 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:32.276943 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:32.276943 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:32.278655 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:32.278655 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:32.278655 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:32.278655 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:32.278655 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:32.278655 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:32.278655 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:32.278655 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:32.278655 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:32.278655 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:32.278655 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:32.278655 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:32.278655 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:32.278655 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:32.278655 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:32.278655 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:32.280179 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:32.280179 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:32.280179 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:32.280179 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:32.280179 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:32.280179 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:32.280179 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:32.280179 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:32.280179 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:32.280179 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:32.280179 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:49:32.280179 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:32.280179 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 163/329] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:32.286109 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73444dc4-84cb-4b69-ab38-0e4e51f6570c None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:32.332165 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:32.338448 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61e0da-18fe-41e5-afc5-1f12aea09612 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:32.373677 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:32.387876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853eb8b1-dcf4-4594-828b-f02e15e325db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:32.388532 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853eb8b1-dcf4-4594-828b-f02e15e325db None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.388854 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853eb8b1-dcf4-4594-828b-f02e15e325db None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.389297 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853eb8b1-dcf4-4594-828b-f02e15e325db None admin] PATH_INFO: `/v3/users/379585e2a798420caa7f47df7fddadbd` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.391693 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853eb8b1-dcf4-4594-828b-f02e15e325db None admin] RBAC: Authorizing `identity:get_user(user_id=379585e2a798420caa7f47df7fddadbd)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:32.393245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853eb8b1-dcf4-4594-828b-f02e15e325db None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:32.394951 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 167/330] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:32 2022] GET /identity/v3/users/379585e2a798420caa7f47df7fddadbd => generated 293 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:32.404851 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853eb8b1-dcf4-4594-828b-f02e15e325db None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:32.429739 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73444dc4-84cb-4b69-ab38-0e4e51f6570c None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:32.488615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44217b77-c1a5-47ab-8604-7d090a463b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:32.489581 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44217b77-c1a5-47ab-8604-7d090a463b30 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.490003 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44217b77-c1a5-47ab-8604-7d090a463b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.490432 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44217b77-c1a5-47ab-8604-7d090a463b30 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.491635 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44217b77-c1a5-47ab-8604-7d090a463b30 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:32.492726 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44217b77-c1a5-47ab-8604-7d090a463b30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:32.494538 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44217b77-c1a5-47ab-8604-7d090a463b30 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:32.507816 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 164/331] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:32.513900 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44217b77-c1a5-47ab-8604-7d090a463b30 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:32.535234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853eb8b1-dcf4-4594-828b-f02e15e325db None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:32.593845 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc8639-f337-40ce-8d26-7cf61bf877b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:32.594768 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc8639-f337-40ce-8d26-7cf61bf877b5 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.595074 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc8639-f337-40ce-8d26-7cf61bf877b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.595358 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc8639-f337-40ce-8d26-7cf61bf877b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.596169 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc8639-f337-40ce-8d26-7cf61bf877b5 None admin] RBAC: Adding query filter params (role.id=fbd7c81b4fba4b6d9e61b1e9bb3e2046, user.id=379585e2a798420caa7f47df7fddadbd, scope.system=all) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:32.596489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc8639-f337-40ce-8d26-7cf61bf877b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:32.598839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc8639-f337-40ce-8d26-7cf61bf877b5 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:32.619295 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 168/332] 10.222.0.39 () {56 vars in 1308 bytes} [Fri Aug 5 19:49:32 2022] GET /identity/v3/role_assignments?user.id=379585e2a798420caa7f47df7fddadbd&scope.system=all&role.id=fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 510 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:32.676138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:32.683410 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44217b77-c1a5-47ab-8604-7d090a463b30 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:32.726663 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f0245-4423-40b1-9fc2-2365fed487b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:32.727690 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f0245-4423-40b1-9fc2-2365fed487b0 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.728121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f0245-4423-40b1-9fc2-2365fed487b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.728564 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f0245-4423-40b1-9fc2-2365fed487b0 None admin] PATH_INFO: `/v3/users/2a793618c68a49fb84c23d606d32fed1` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.730299 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f0245-4423-40b1-9fc2-2365fed487b0 None admin] RBAC: Authorizing `identity:get_user(user_id=2a793618c68a49fb84c23d606d32fed1)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:32.731970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f0245-4423-40b1-9fc2-2365fed487b0 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:32.734869 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 165/333] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:32 2022] GET /identity/v3/users/2a793618c68a49fb84c23d606d32fed1 => generated 293 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:32.741773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc8639-f337-40ce-8d26-7cf61bf877b5 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:32.810201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:32.875178 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc8639-f337-40ce-8d26-7cf61bf877b5 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:32.917018 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658440e-e6d7-4a38-8a85-52cb40483994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:32.917689 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658440e-e6d7-4a38-8a85-52cb40483994 None admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.917972 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658440e-e6d7-4a38-8a85-52cb40483994 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.918246 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658440e-e6d7-4a38-8a85-52cb40483994 None admin] PATH_INFO: `/v3/system/users/2a793618c68a49fb84c23d606d32fed1/roles/9de09806c19c49e7aba24d414a65eb5f` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.920968 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9658440e-e6d7-4a38-8a85-52cb40483994 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2a793618c68a49fb84c23d606d32fed1, role_id=9de09806c19c49e7aba24d414a65eb5f)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:32.922782 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9658440e-e6d7-4a38-8a85-52cb40483994 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:32.932788 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 169/334] 10.222.0.39 () {58 vars in 1334 bytes} [Fri Aug 5 19:49:32 2022] PUT /identity/v3/system/users/2a793618c68a49fb84c23d606d32fed1/roles/9de09806c19c49e7aba24d414a65eb5f => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:34.930957 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdb1a4-278b-4424-ad4f-c94c45e1d719 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.931515 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdb1a4-278b-4424-ad4f-c94c45e1d719 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.931515 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdb1a4-278b-4424-ad4f-c94c45e1d719 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.933478 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 166/335] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:34.940867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d07aa-4a19-4475-b1ca-d0015eaec8dc None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.941465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d07aa-4a19-4475-b1ca-d0015eaec8dc None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.941465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d07aa-4a19-4475-b1ca-d0015eaec8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.969988 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-6e2d07aa-4a19-4475-b1ca-d0015eaec8dc None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:34.971216 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-6e2d07aa-4a19-4475-b1ca-d0015eaec8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:35.064046 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 170/336] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:35.120101 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fdb1a4-278b-4424-ad4f-c94c45e1d719 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:35.367820 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fdb1a4-278b-4424-ad4f-c94c45e1d719 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:35.445195 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a92319-920d-49ef-bc34-7aefdb5690bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:35.446253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a92319-920d-49ef-bc34-7aefdb5690bc None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:35.446743 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a92319-920d-49ef-bc34-7aefdb5690bc None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:35.447222 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a92319-920d-49ef-bc34-7aefdb5690bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:35.472782 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb1f1b-b695-4e43-815b-f34486561bf6 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:35.473122 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb1f1b-b695-4e43-815b-f34486561bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:35.473343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb1f1b-b695-4e43-815b-f34486561bf6 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:35.474968 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 171/337] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:35.479683 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd94fd2-0e68-4928-a60b-485f853932ee None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:35.479969 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd94fd2-0e68-4928-a60b-485f853932ee None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:35.480249 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd94fd2-0e68-4928-a60b-485f853932ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:35.484598 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a92319-920d-49ef-bc34-7aefdb5690bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:35.495435 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-56a92319-920d-49ef-bc34-7aefdb5690bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:35.495435 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:35.495435 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:35.495435 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:35.495435 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:35.495435 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:35.495435 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:35.495435 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:35.495435 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:35.495435 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:35.495435 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:35.495435 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:35.495435 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:35.495435 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:35.495435 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:35.495435 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:35.496706 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:35.496706 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:35.496706 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:35.496706 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:35.496706 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:35.496706 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:35.496706 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:35.496706 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:35.496706 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:35.496706 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:35.496706 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:35.496706 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:35.496706 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:35.496706 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:35.496706 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:35.496706 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:35.497521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:35.497521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:35.497521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:35.497521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:35.497521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:35.497521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:35.497521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:35.497521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:35.497521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:35.497521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:35.497521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:35.497521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:35.498680 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 167/338] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:35.505889 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a92319-920d-49ef-bc34-7aefdb5690bc None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:35.516028 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-2fd94fd2-0e68-4928-a60b-485f853932ee None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:35.517331 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-2fd94fd2-0e68-4928-a60b-485f853932ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:35.589999 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:35.603328 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:35.660534 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 172/339] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:35.698320 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd94fd2-0e68-4928-a60b-485f853932ee None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:35.765866 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a92319-920d-49ef-bc34-7aefdb5690bc None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:35.841347 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b709959-6dda-4188-8f65-240dd1033d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:35.842391 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b709959-6dda-4188-8f65-240dd1033d37 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:35.843096 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b709959-6dda-4188-8f65-240dd1033d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:35.843580 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b709959-6dda-4188-8f65-240dd1033d37 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:35.844593 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b709959-6dda-4188-8f65-240dd1033d37 None admin] RBAC: Adding query filter params (name=member) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:35.845110 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b709959-6dda-4188-8f65-240dd1033d37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:35.846723 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b709959-6dda-4188-8f65-240dd1033d37 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:35.875597 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 168/340] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:35.877750 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a1518-660e-4f0e-bb22-73f71ebd2261 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:35.878247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a1518-660e-4f0e-bb22-73f71ebd2261 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:35.878752 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a1518-660e-4f0e-bb22-73f71ebd2261 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:35.885390 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 169/341] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:35.886877 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a1518-660e-4f0e-bb22-73f71ebd2261 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:35.962375 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd94fd2-0e68-4928-a60b-485f853932ee None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:36.054528 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b0ee6-ff5f-4111-97a2-aec89da276bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:36.055194 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b0ee6-ff5f-4111-97a2-aec89da276bc None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:36.055476 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b0ee6-ff5f-4111-97a2-aec89da276bc None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:36.055751 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b0ee6-ff5f-4111-97a2-aec89da276bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:36.080988 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b0ee6-ff5f-4111-97a2-aec89da276bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:36.086055 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-f36b0ee6-ff5f-4111-97a2-aec89da276bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:36.086055 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:36.086055 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:36.086055 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:36.086055 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:36.086055 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:36.086055 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:36.086055 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:36.086055 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:36.086055 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:36.086055 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:36.086055 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:36.086055 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:36.086055 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:36.086055 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:36.086055 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:36.086942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:36.086942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:36.086942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:36.086942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:36.086942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:36.086942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:36.086942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:36.086942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:36.086942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:36.086942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:36.086942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:36.086942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:36.086942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:36.086942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:36.086942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:36.086942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:36.088879 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:36.088879 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:36.088879 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:36.088879 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:36.088879 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:36.088879 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:36.088879 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:36.088879 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:36.088879 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:36.088879 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:36.088879 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:36.088879 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:49:36.088879 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 173/342] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:36.088879 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b167f4-e5f9-4ece-bb9e-61c7e7eddd1b None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:36.089773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b167f4-e5f9-4ece-bb9e-61c7e7eddd1b None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:36.089773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b167f4-e5f9-4ece-bb9e-61c7e7eddd1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:36.114919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-83b167f4-e5f9-4ece-bb9e-61c7e7eddd1b None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:36.114919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-83b167f4-e5f9-4ece-bb9e-61c7e7eddd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:36.139965 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:36.157721 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:36.185043 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a1518-660e-4f0e-bb22-73f71ebd2261 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:36.229549 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 174/343] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:36.230108 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b167f4-e5f9-4ece-bb9e-61c7e7eddd1b None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:36.259851 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966b984-7087-4a07-9b38-a51b7c32d751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:36.262030 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966b984-7087-4a07-9b38-a51b7c32d751 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:36.262313 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966b984-7087-4a07-9b38-a51b7c32d751 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:36.262587 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966b984-7087-4a07-9b38-a51b7c32d751 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:36.416344 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6966b984-7087-4a07-9b38-a51b7c32d751 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:36.417864 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6966b984-7087-4a07-9b38-a51b7c32d751 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:36.459429 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b167f4-e5f9-4ece-bb9e-61c7e7eddd1b None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:36.526697 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5da08a-1d15-4cab-a922-384ba0021033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:36.527254 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5da08a-1d15-4cab-a922-384ba0021033 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:36.527534 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5da08a-1d15-4cab-a922-384ba0021033 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:36.527809 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5da08a-1d15-4cab-a922-384ba0021033 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:36.528536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5da08a-1d15-4cab-a922-384ba0021033 None admin] RBAC: Adding query filter params (name=member) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:36.528885 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5da08a-1d15-4cab-a922-384ba0021033 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:36.530441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5da08a-1d15-4cab-a922-384ba0021033 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:36.540262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 175/344] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:36.541215 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5da08a-1d15-4cab-a922-384ba0021033 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:36.541953 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:36.563663 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 170/345] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 678 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:36.565214 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b60d3-a72e-4425-b22e-ef472e007cb1 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:36.565761 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b60d3-a72e-4425-b22e-ef472e007cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:36.566215 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b60d3-a72e-4425-b22e-ef472e007cb1 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:36.568014 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 171/346] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:36.568918 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b60d3-a72e-4425-b22e-ef472e007cb1 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:36.582476 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:36.675386 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5da08a-1d15-4cab-a922-384ba0021033 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:36.704242 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b60d3-a72e-4425-b22e-ef472e007cb1 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:36.725717 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2088a4-eddc-475f-8ca0-85ba85b160ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:36.726225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2088a4-eddc-475f-8ca0-85ba85b160ae None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:36.726508 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2088a4-eddc-475f-8ca0-85ba85b160ae None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:36.726782 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2088a4-eddc-475f-8ca0-85ba85b160ae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:36.742283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2088a4-eddc-475f-8ca0-85ba85b160ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:36.750554 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-de2088a4-eddc-475f-8ca0-85ba85b160ae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:36.750554 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:36.750554 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:36.750554 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:36.750554 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:36.750554 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:36.750554 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:36.750554 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:36.750554 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:36.750554 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:36.750554 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:36.750554 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:36.750554 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:36.750554 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:36.750554 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:36.750554 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:36.751546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:36.751546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:36.751546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:36.751546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:36.751546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:36.751546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:36.751546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:36.751546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:36.751546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:36.751546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:36.751546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:36.751546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:36.751546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:36.751546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:36.751546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:36.751546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:36.752443 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:36.752443 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:36.752443 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:36.752443 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:36.752443 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:36.752443 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:36.752443 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:36.752443 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:36.752443 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:36.752443 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:36.752443 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:36.752443 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:49:36.752443 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 176/347] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Aug 5 19:49:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:36.753654 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2088a4-eddc-475f-8ca0-85ba85b160ae None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:36.770826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234c9d4-e427-4215-ac92-16f35b45c0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:36.771454 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234c9d4-e427-4215-ac92-16f35b45c0e0 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:36.771731 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234c9d4-e427-4215-ac92-16f35b45c0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:36.772001 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234c9d4-e427-4215-ac92-16f35b45c0e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:36.846142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:36.873727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c234c9d4-e427-4215-ac92-16f35b45c0e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:36.876505 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c234c9d4-e427-4215-ac92-16f35b45c0e0 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:36.881628 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:36.910715 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2088a4-eddc-475f-8ca0-85ba85b160ae None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:36.967109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a24f929-1a19-49e9-be9c-fa0d90c4d048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:36.968997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24f929-1a19-49e9-be9c-fa0d90c4d048 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:36.969310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24f929-1a19-49e9-be9c-fa0d90c4d048 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:36.969919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24f929-1a19-49e9-be9c-fa0d90c4d048 None admin] PATH_INFO: `/v3/projects/f542d45b34d849d2aad08bc75ebc275a` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:36.974792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a24f929-1a19-49e9-be9c-fa0d90c4d048 None admin] RBAC: Authorizing `identity:get_project(project_id=f542d45b34d849d2aad08bc75ebc275a)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:36.976162 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a24f929-1a19-49e9-be9c-fa0d90c4d048 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:36.979557 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 177/348] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:36 2022] GET /identity/v3/projects/f542d45b34d849d2aad08bc75ebc275a => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:36.981134 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f200e-225f-41d9-811c-acd3fc021ce1 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:36.981558 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f200e-225f-41d9-811c-acd3fc021ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:36.981895 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f200e-225f-41d9-811c-acd3fc021ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.012131 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-7c6f200e-225f-41d9-811c-acd3fc021ce1 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:37.013957 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-7c6f200e-225f-41d9-811c-acd3fc021ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:37.065112 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 172/349] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:37.065112 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c474c-4596-4ec9-ad52-071010ca7934 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:37.065112 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c474c-4596-4ec9-ad52-071010ca7934 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:37.065452 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c474c-4596-4ec9-ad52-071010ca7934 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.067323 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 173/350] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:37.072076 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595c474c-4596-4ec9-ad52-071010ca7934 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:37.130352 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 178/351] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:37.131833 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f200e-225f-41d9-811c-acd3fc021ce1 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:37.227750 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:37.251058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595c474c-4596-4ec9-ad52-071010ca7934 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:37.273439 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:37.294263 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f200e-225f-41d9-811c-acd3fc021ce1 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:37.308188 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ea85b-df21-4dc0-8513-28a98402f930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:37.309681 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ea85b-df21-4dc0-8513-28a98402f930 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:37.310067 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ea85b-df21-4dc0-8513-28a98402f930 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:37.310352 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ea85b-df21-4dc0-8513-28a98402f930 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.311106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629ea85b-df21-4dc0-8513-28a98402f930 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:37.311466 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629ea85b-df21-4dc0-8513-28a98402f930 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:37.312939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629ea85b-df21-4dc0-8513-28a98402f930 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:37.322011 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 174/352] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:37.322724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ea85b-df21-4dc0-8513-28a98402f930 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:37.352420 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f05dda2-e1a4-4591-951c-ca4ccdb4da60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:37.353237 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f05dda2-e1a4-4591-951c-ca4ccdb4da60 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:37.353520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f05dda2-e1a4-4591-951c-ca4ccdb4da60 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:37.353794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f05dda2-e1a4-4591-951c-ca4ccdb4da60 None admin] PATH_INFO: `/v3/groups/f2e4c5e7f5df4798982e58481f941bfe` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.364999 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f05dda2-e1a4-4591-951c-ca4ccdb4da60 None admin] RBAC: Authorizing `identity:get_group(group_id=f2e4c5e7f5df4798982e58481f941bfe)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:37.366596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f05dda2-e1a4-4591-951c-ca4ccdb4da60 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:37.369552 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 179/353] 10.222.0.39 () {56 vars in 1180 bytes} [Fri Aug 5 19:49:37 2022] GET /identity/v3/groups/f2e4c5e7f5df4798982e58481f941bfe => generated 226 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:37.370880 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc6125-2de6-4922-bd1f-7764560995ae None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:37.371201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc6125-2de6-4922-bd1f-7764560995ae None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:37.371466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc6125-2de6-4922-bd1f-7764560995ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.392848 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-c4cc6125-2de6-4922-bd1f-7764560995ae None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:37.393957 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-c4cc6125-2de6-4922-bd1f-7764560995ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:37.482392 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ea85b-df21-4dc0-8513-28a98402f930 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:37.508600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 180/354] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:37.508600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d90fc-4d46-4879-841f-fdb7736aa57c None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:37.508600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d90fc-4d46-4879-841f-fdb7736aa57c None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:37.508600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d90fc-4d46-4879-841f-fdb7736aa57c None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.508600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 181/355] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:37.508600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d90fc-4d46-4879-841f-fdb7736aa57c None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:37.531224 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4d246-1f79-4161-8bbf-a19cefd57f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:37.533090 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4d246-1f79-4161-8bbf-a19cefd57f1a None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:37.533604 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4d246-1f79-4161-8bbf-a19cefd57f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:37.534426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4d246-1f79-4161-8bbf-a19cefd57f1a None admin] PATH_INFO: `/v3/projects/c0ded24d8148467b9a56fa55332feb56` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.536841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b4d246-1f79-4161-8bbf-a19cefd57f1a None admin] RBAC: Authorizing `identity:get_project(project_id=c0ded24d8148467b9a56fa55332feb56)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:37.538351 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b4d246-1f79-4161-8bbf-a19cefd57f1a None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:37.540436 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 175/356] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:37 2022] GET /identity/v3/projects/c0ded24d8148467b9a56fa55332feb56 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:37.541543 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4d246-1f79-4161-8bbf-a19cefd57f1a None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:37.564497 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:37.653418 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:37.680329 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4d246-1f79-4161-8bbf-a19cefd57f1a None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:37.680849 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d90fc-4d46-4879-841f-fdb7736aa57c None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:37.728563 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ba7f8-28db-45eb-b92b-182954915606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:37.729226 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ba7f8-28db-45eb-b92b-182954915606 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:37.729513 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ba7f8-28db-45eb-b92b-182954915606 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:37.730064 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ba7f8-28db-45eb-b92b-182954915606 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.738180 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb140d-74ea-4343-8402-392b0746a9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:37.738835 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb140d-74ea-4343-8402-392b0746a9a6 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:37.739112 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb140d-74ea-4343-8402-392b0746a9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:37.739378 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb140d-74ea-4343-8402-392b0746a9a6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.759317 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bb140d-74ea-4343-8402-392b0746a9a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:37.766163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-a2bb140d-74ea-4343-8402-392b0746a9a6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:37.766163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:37.766163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:37.766163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:37.766163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:37.766163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:37.766163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:37.766163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:37.766163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:37.766163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:37.766163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:37.766163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:37.766163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:37.766163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:37.766163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:37.766163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:37.767396 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:37.767396 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:37.767396 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:37.767396 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:37.767396 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:37.767396 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:37.767396 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:37.767396 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:37.767396 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:37.767396 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:37.767396 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:37.767396 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:37.767396 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:37.767396 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:37.767396 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:37.767396 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:37.769104 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:37.769104 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:37.769104 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:37.769104 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:37.769104 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:37.769104 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:37.769104 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:37.769104 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:37.769104 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:37.769104 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:37.769104 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:37.769104 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:49:37.769104 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 182/357] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Aug 5 19:49:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:37.769104 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb140d-74ea-4343-8402-392b0746a9a6 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:37.844012 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ba7f8-28db-45eb-b92b-182954915606 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:37.845512 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ba7f8-28db-45eb-b92b-182954915606 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:37.866800 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:37.915275 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb140d-74ea-4343-8402-392b0746a9a6 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:37.958644 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811d5a5-f100-40e3-942a-07bca253916f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:37.959266 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811d5a5-f100-40e3-942a-07bca253916f None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:37.959544 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811d5a5-f100-40e3-942a-07bca253916f None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:37.959813 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811d5a5-f100-40e3-942a-07bca253916f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.960674 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a811d5a5-f100-40e3-942a-07bca253916f None admin] RBAC: Adding query filter params (group.id=f2e4c5e7f5df4798982e58481f941bfe, role.id=fbd7c81b4fba4b6d9e61b1e9bb3e2046, scope.project.id=f542d45b34d849d2aad08bc75ebc275a) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:37.960954 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a811d5a5-f100-40e3-942a-07bca253916f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:37.963125 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a811d5a5-f100-40e3-942a-07bca253916f None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:37.971202 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 183/358] 10.222.0.39 () {56 vars in 1376 bytes} [Fri Aug 5 19:49:37 2022] GET /identity/v3/role_assignments?group.id=f2e4c5e7f5df4798982e58481f941bfe&scope.project.id=f542d45b34d849d2aad08bc75ebc275a&role.id=fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:37.972451 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e17a6-7255-4dd1-8fbe-5f74cac299e0 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:37.972785 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e17a6-7255-4dd1-8fbe-5f74cac299e0 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:37.973062 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e17a6-7255-4dd1-8fbe-5f74cac299e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.980660 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:37.997175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 176/359] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:37.998074 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ba7f8-28db-45eb-b92b-182954915606 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:38.000540 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-5b7e17a6-7255-4dd1-8fbe-5f74cac299e0 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:38.002305 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5b7e17a6-7255-4dd1-8fbe-5f74cac299e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:38.107510 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 184/360] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:38.113166 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e17a6-7255-4dd1-8fbe-5f74cac299e0 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:38.168149 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ba7f8-28db-45eb-b92b-182954915606 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:38.200094 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:38.235570 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18859cb6-861b-4d68-aae7-2f2fccf27343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:38.237177 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18859cb6-861b-4d68-aae7-2f2fccf27343 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:38.237177 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18859cb6-861b-4d68-aae7-2f2fccf27343 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:38.237424 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18859cb6-861b-4d68-aae7-2f2fccf27343 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:38.256873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18859cb6-861b-4d68-aae7-2f2fccf27343 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:38.262942 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-18859cb6-861b-4d68-aae7-2f2fccf27343 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:49:38.262942 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:38.262942 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:38.262942 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:38.262942 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:38.262942 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:38.262942 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:38.262942 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:38.262942 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:38.262942 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:38.262942 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:38.262942 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:38.262942 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:38.262942 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:38.262942 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:38.262942 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:38.263970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:38.263970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:38.263970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:38.263970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:38.263970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:38.263970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:38.263970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:38.263970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:38.263970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:38.263970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:38.263970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:38.263970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:38.263970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:38.263970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:38.263970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:38.263970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:38.265296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:38.265296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:38.265296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:38.265296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:38.265296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:38.265296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:38.265296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:38.265296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:38.265296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:38.265296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:38.265296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:49:38.265296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:38.265296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 177/361] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:38.266391 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18859cb6-861b-4d68-aae7-2f2fccf27343 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:38.279492 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e17a6-7255-4dd1-8fbe-5f74cac299e0 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:38.331200 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d4b92-ac4f-4d2e-84c3-f40bc6c111b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:38.331847 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d4b92-ac4f-4d2e-84c3-f40bc6c111b0 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:38.332169 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d4b92-ac4f-4d2e-84c3-f40bc6c111b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:38.332444 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d4b92-ac4f-4d2e-84c3-f40bc6c111b0 None admin] PATH_INFO: `/v3/groups/f2e4c5e7f5df4798982e58481f941bfe` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:38.333645 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d4b92-ac4f-4d2e-84c3-f40bc6c111b0 None admin] RBAC: Authorizing `identity:get_group(group_id=f2e4c5e7f5df4798982e58481f941bfe)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:38.335272 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d4b92-ac4f-4d2e-84c3-f40bc6c111b0 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:38.337121 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 185/362] 10.222.0.39 () {56 vars in 1180 bytes} [Fri Aug 5 19:49:38 2022] GET /identity/v3/groups/f2e4c5e7f5df4798982e58481f941bfe => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:38.337908 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d4b92-ac4f-4d2e-84c3-f40bc6c111b0 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:38.340956 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:38.425494 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18859cb6-861b-4d68-aae7-2f2fccf27343 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:38.473672 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae80f0-d5b5-42bd-8544-c14dcc021715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:38.474397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae80f0-d5b5-42bd-8544-c14dcc021715 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:38.474759 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae80f0-d5b5-42bd-8544-c14dcc021715 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:38.475208 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae80f0-d5b5-42bd-8544-c14dcc021715 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:38.476098 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae80f0-d5b5-42bd-8544-c14dcc021715 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:38.476475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae80f0-d5b5-42bd-8544-c14dcc021715 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:38.478043 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae80f0-d5b5-42bd-8544-c14dcc021715 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:38.485020 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d4b92-ac4f-4d2e-84c3-f40bc6c111b0 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:38.490820 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 178/363] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:38.491762 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae80f0-d5b5-42bd-8544-c14dcc021715 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:38.543601 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7906abb1-05a7-43fa-bb5e-3a130929e31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:38.544281 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906abb1-05a7-43fa-bb5e-3a130929e31a None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:38.544567 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906abb1-05a7-43fa-bb5e-3a130929e31a None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:38.544890 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906abb1-05a7-43fa-bb5e-3a130929e31a None admin] PATH_INFO: `/v3/projects/f542d45b34d849d2aad08bc75ebc275a` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:38.546164 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7906abb1-05a7-43fa-bb5e-3a130929e31a None admin] RBAC: Authorizing `identity:get_project(project_id=f542d45b34d849d2aad08bc75ebc275a)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:38.547790 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7906abb1-05a7-43fa-bb5e-3a130929e31a None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:38.554144 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 186/364] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:38 2022] GET /identity/v3/projects/f542d45b34d849d2aad08bc75ebc275a => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:38.554144 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7906abb1-05a7-43fa-bb5e-3a130929e31a None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:38.569419 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:38.646811 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:38.650314 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae80f0-d5b5-42bd-8544-c14dcc021715 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:38.695935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcdc29b-62da-48ef-bb15-82447289408f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:38.696559 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcdc29b-62da-48ef-bb15-82447289408f None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:38.696804 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcdc29b-62da-48ef-bb15-82447289408f None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:38.697079 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcdc29b-62da-48ef-bb15-82447289408f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:38.698405 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7906abb1-05a7-43fa-bb5e-3a130929e31a None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:38.710989 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcdc29b-62da-48ef-bb15-82447289408f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:38.717175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-cdcdc29b-62da-48ef-bb15-82447289408f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:49:38.717175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:38.717175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:38.717175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:38.717175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:38.717175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:38.717175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:38.717175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:38.717175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:38.717175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:38.717175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:38.717175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:38.717175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:38.717175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:38.717175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:38.717175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:38.718724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:38.718724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:38.718724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:38.718724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:38.718724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:38.718724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:38.718724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:38.718724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:38.718724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:38.718724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:38.718724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:38.718724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:38.718724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:38.718724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:38.718724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:38.718724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:38.719505 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:38.719505 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:38.719505 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:38.719505 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:38.719505 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:38.719505 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:38.719505 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:38.719505 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:38.719505 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:38.719505 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:38.719505 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:49:38.719505 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:38.719505 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 179/365] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:49:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:38.720201 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcdc29b-62da-48ef-bb15-82447289408f None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:38.744133 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd6be8c-a800-438a-bbcf-ad46ae6c5523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:38.744802 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd6be8c-a800-438a-bbcf-ad46ae6c5523 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:38.745086 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd6be8c-a800-438a-bbcf-ad46ae6c5523 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:38.745360 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd6be8c-a800-438a-bbcf-ad46ae6c5523 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:38.746089 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd6be8c-a800-438a-bbcf-ad46ae6c5523 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:38.746401 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd6be8c-a800-438a-bbcf-ad46ae6c5523 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:38.748053 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd6be8c-a800-438a-bbcf-ad46ae6c5523 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:38.757575 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 187/366] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:38.758434 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd6be8c-a800-438a-bbcf-ad46ae6c5523 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:38.851431 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:38.859735 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcdc29b-62da-48ef-bb15-82447289408f None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:38.899427 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd6be8c-a800-438a-bbcf-ad46ae6c5523 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:38.901139 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83acd3d-dfd1-4e3c-ab49-d1506710c5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:38.901837 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83acd3d-dfd1-4e3c-ab49-d1506710c5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:38.902123 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83acd3d-dfd1-4e3c-ab49-d1506710c5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:38.902598 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83acd3d-dfd1-4e3c-ab49-d1506710c5c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:38.903398 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83acd3d-dfd1-4e3c-ab49-d1506710c5c3 None admin] RBAC: Adding query filter params (group.id=f2e4c5e7f5df4798982e58481f941bfe, role.id=fbd7c81b4fba4b6d9e61b1e9bb3e2046, scope.project.id=c0ded24d8148467b9a56fa55332feb56) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:38.904600 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83acd3d-dfd1-4e3c-ab49-d1506710c5c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:38.906796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83acd3d-dfd1-4e3c-ab49-d1506710c5c3 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:38.920632 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 180/367] 10.222.0.39 () {56 vars in 1376 bytes} [Fri Aug 5 19:49:38 2022] GET /identity/v3/role_assignments?group.id=f2e4c5e7f5df4798982e58481f941bfe&scope.project.id=c0ded24d8148467b9a56fa55332feb56&role.id=fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:38.920632 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83acd3d-dfd1-4e3c-ab49-d1506710c5c3 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:38.953689 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:38.959909 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf76f61-1ac6-4c8a-ad52-c7587b00ea47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:38.960807 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf76f61-1ac6-4c8a-ad52-c7587b00ea47 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:38.961227 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf76f61-1ac6-4c8a-ad52-c7587b00ea47 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:38.961520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf76f61-1ac6-4c8a-ad52-c7587b00ea47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:39.054847 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf76f61-1ac6-4c8a-ad52-c7587b00ea47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:39.057309 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf76f61-1ac6-4c8a-ad52-c7587b00ea47 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:39.080761 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83acd3d-dfd1-4e3c-ab49-d1506710c5c3 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:39.128498 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513ae6e-8c89-496e-b84b-4233409fa4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:39.129232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513ae6e-8c89-496e-b84b-4233409fa4cf None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:39.129474 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513ae6e-8c89-496e-b84b-4233409fa4cf None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:39.129869 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513ae6e-8c89-496e-b84b-4233409fa4cf None admin] PATH_INFO: `/v3/groups/f2e4c5e7f5df4798982e58481f941bfe` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:39.132925 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a513ae6e-8c89-496e-b84b-4233409fa4cf None admin] RBAC: Authorizing `identity:get_group(group_id=f2e4c5e7f5df4798982e58481f941bfe)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:39.135609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a513ae6e-8c89-496e-b84b-4233409fa4cf None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:39.139041 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 181/368] 10.222.0.39 () {56 vars in 1180 bytes} [Fri Aug 5 19:49:38 2022] GET /identity/v3/groups/f2e4c5e7f5df4798982e58481f941bfe => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:39.144241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513ae6e-8c89-496e-b84b-4233409fa4cf None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:39.190835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:39.199676 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 188/369] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:39.200686 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf76f61-1ac6-4c8a-ad52-c7587b00ea47 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:39.245837 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:39.289792 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513ae6e-8c89-496e-b84b-4233409fa4cf None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:39.335965 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf76f61-1ac6-4c8a-ad52-c7587b00ea47 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:39.344063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee369f-c6e2-4637-88d2-8a068e79efb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:39.344715 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee369f-c6e2-4637-88d2-8a068e79efb9 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:39.344995 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee369f-c6e2-4637-88d2-8a068e79efb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:39.345326 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee369f-c6e2-4637-88d2-8a068e79efb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:39.346046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee369f-c6e2-4637-88d2-8a068e79efb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:39.346402 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee369f-c6e2-4637-88d2-8a068e79efb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:39.347826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee369f-c6e2-4637-88d2-8a068e79efb9 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:39.359493 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 182/370] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:39.360195 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee369f-c6e2-4637-88d2-8a068e79efb9 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:39.384081 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b3b438-181e-4d5b-80b7-59eb836cae50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:39.385042 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3b438-181e-4d5b-80b7-59eb836cae50 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:39.385465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3b438-181e-4d5b-80b7-59eb836cae50 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:39.385901 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3b438-181e-4d5b-80b7-59eb836cae50 None admin] PATH_INFO: `/v3/users/2a793618c68a49fb84c23d606d32fed1` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:39.387741 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b3b438-181e-4d5b-80b7-59eb836cae50 None admin] RBAC: Authorizing `identity:get_user(user_id=2a793618c68a49fb84c23d606d32fed1)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:39.391498 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b3b438-181e-4d5b-80b7-59eb836cae50 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:39.399038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 189/371] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:39 2022] GET /identity/v3/users/2a793618c68a49fb84c23d606d32fed1 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:39.399038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b3b438-181e-4d5b-80b7-59eb836cae50 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:39.510683 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:39.524119 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee369f-c6e2-4637-88d2-8a068e79efb9 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:39.529784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:39.533695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b3b438-181e-4d5b-80b7-59eb836cae50 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:39.569279 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c34a900-99db-48a7-9f9a-279e06305578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:39.571375 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34a900-99db-48a7-9f9a-279e06305578 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:39.571920 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34a900-99db-48a7-9f9a-279e06305578 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:39.572643 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34a900-99db-48a7-9f9a-279e06305578 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:39.573621 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c34a900-99db-48a7-9f9a-279e06305578 None admin] RBAC: Adding query filter params (group.id=f2e4c5e7f5df4798982e58481f941bfe, role.id=0eafd8a7ba2d47faaf5aecbb3360143b, scope.project.id=f542d45b34d849d2aad08bc75ebc275a) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:39.574235 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c34a900-99db-48a7-9f9a-279e06305578 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:39.577020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c34a900-99db-48a7-9f9a-279e06305578 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:39.582905 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e3fd7-c86e-4b64-8444-7cb58b97aaf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:39.583747 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e3fd7-c86e-4b64-8444-7cb58b97aaf5 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:39.584134 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e3fd7-c86e-4b64-8444-7cb58b97aaf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:39.584515 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e3fd7-c86e-4b64-8444-7cb58b97aaf5 None admin] PATH_INFO: `/v3/projects/c0ded24d8148467b9a56fa55332feb56` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:39.586071 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e3fd7-c86e-4b64-8444-7cb58b97aaf5 None admin] RBAC: Authorizing `identity:get_project(project_id=c0ded24d8148467b9a56fa55332feb56)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:39.586286 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 183/372] 10.222.0.39 () {56 vars in 1376 bytes} [Fri Aug 5 19:49:39 2022] GET /identity/v3/role_assignments?group.id=f2e4c5e7f5df4798982e58481f941bfe&scope.project.id=f542d45b34d849d2aad08bc75ebc275a&role.id=0eafd8a7ba2d47faaf5aecbb3360143b => generated 260 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:39.587813 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e3fd7-c86e-4b64-8444-7cb58b97aaf5 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:39.588055 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c34a900-99db-48a7-9f9a-279e06305578 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:39.590235 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 190/373] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:39 2022] GET /identity/v3/projects/c0ded24d8148467b9a56fa55332feb56 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:39.591560 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e3fd7-c86e-4b64-8444-7cb58b97aaf5 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:39.749662 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c34a900-99db-48a7-9f9a-279e06305578 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:39.759981 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e3fd7-c86e-4b64-8444-7cb58b97aaf5 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:39.830130 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fddc7f-abdf-42ca-8552-4944a913be4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:39.830688 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fddc7f-abdf-42ca-8552-4944a913be4e None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:39.830970 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fddc7f-abdf-42ca-8552-4944a913be4e None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:39.831247 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fddc7f-abdf-42ca-8552-4944a913be4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:39.832063 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fddc7f-abdf-42ca-8552-4944a913be4e None admin] RBAC: Adding query filter params (role.id=9de09806c19c49e7aba24d414a65eb5f, user.id=2a793618c68a49fb84c23d606d32fed1, scope.system=all) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:39.832379 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fddc7f-abdf-42ca-8552-4944a913be4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:39.833952 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fddc7f-abdf-42ca-8552-4944a913be4e None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:39.844525 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a774a-817e-4994-8a5d-81b10543f36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:39.845186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a774a-817e-4994-8a5d-81b10543f36f None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:39.845469 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a774a-817e-4994-8a5d-81b10543f36f None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:39.845745 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a774a-817e-4994-8a5d-81b10543f36f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:39.865976 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 191/374] 10.222.0.39 () {56 vars in 1308 bytes} [Fri Aug 5 19:49:39 2022] GET /identity/v3/role_assignments?user.id=2a793618c68a49fb84c23d606d32fed1&scope.system=all&role.id=9de09806c19c49e7aba24d414a65eb5f => generated 510 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:39.867537 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fddc7f-abdf-42ca-8552-4944a913be4e None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:39.905736 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:39.922798 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:40.007979 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fddc7f-abdf-42ca-8552-4944a913be4e None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:40.015022 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1a774a-817e-4994-8a5d-81b10543f36f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:40.017388 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1a774a-817e-4994-8a5d-81b10543f36f None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:40.048372 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3544e-bd52-49c8-b304-34ca4cc822e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:40.049692 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3544e-bd52-49c8-b304-34ca4cc822e6 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:40.050176 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3544e-bd52-49c8-b304-34ca4cc822e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:40.050623 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3544e-bd52-49c8-b304-34ca4cc822e6 None admin] PATH_INFO: `/v3/groups/f2e4c5e7f5df4798982e58481f941bfe` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:40.052278 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3544e-bd52-49c8-b304-34ca4cc822e6 None admin] RBAC: Authorizing `identity:get_group(group_id=f2e4c5e7f5df4798982e58481f941bfe)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:40.053990 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3544e-bd52-49c8-b304-34ca4cc822e6 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:40.055821 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 192/375] 10.222.0.39 () {56 vars in 1180 bytes} [Fri Aug 5 19:49:39 2022] GET /identity/v3/groups/f2e4c5e7f5df4798982e58481f941bfe => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:40.062272 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3544e-bd52-49c8-b304-34ca4cc822e6 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:40.242198 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 184/376] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 655 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:40.247987 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a774a-817e-4994-8a5d-81b10543f36f None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:40.288288 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:40.301727 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3544e-bd52-49c8-b304-34ca4cc822e6 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:40.347970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:40.352335 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be8335e-5010-478e-a84c-41135f80591d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:40.353072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8335e-5010-478e-a84c-41135f80591d None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:40.353363 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8335e-5010-478e-a84c-41135f80591d None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:40.353646 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8335e-5010-478e-a84c-41135f80591d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:40.354653 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be8335e-5010-478e-a84c-41135f80591d None admin] RBAC: Adding query filter params (group.id=f2e4c5e7f5df4798982e58481f941bfe, role.id=0eafd8a7ba2d47faaf5aecbb3360143b, scope.project.id=c0ded24d8148467b9a56fa55332feb56) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:40.354814 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be8335e-5010-478e-a84c-41135f80591d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:40.358638 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be8335e-5010-478e-a84c-41135f80591d None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:40.365681 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 193/377] 10.222.0.39 () {56 vars in 1376 bytes} [Fri Aug 5 19:49:40 2022] GET /identity/v3/role_assignments?group.id=f2e4c5e7f5df4798982e58481f941bfe&scope.project.id=c0ded24d8148467b9a56fa55332feb56&role.id=0eafd8a7ba2d47faaf5aecbb3360143b => generated 260 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:40.367030 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d343ac-e607-4458-bb16-f463a19b7d3d None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:40.367321 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d343ac-e607-4458-bb16-f463a19b7d3d None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:40.367597 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d343ac-e607-4458-bb16-f463a19b7d3d None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:40.369026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 194/378] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:40.378190 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4f473-0c9c-43d9-ba75-9c2c56f26a6c None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:40.378722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4f473-0c9c-43d9-ba75-9c2c56f26a6c None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:40.379140 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4f473-0c9c-43d9-ba75-9c2c56f26a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:40.407860 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-78a4f473-0c9c-43d9-ba75-9c2c56f26a6c None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:40.409734 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-78a4f473-0c9c-43d9-ba75-9c2c56f26a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:40.426912 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a774a-817e-4994-8a5d-81b10543f36f None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:40.490706 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46490151-04f1-4ceb-bd4b-bdbde7532ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:40.491373 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46490151-04f1-4ceb-bd4b-bdbde7532ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:40.491690 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46490151-04f1-4ceb-bd4b-bdbde7532ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:40.492040 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46490151-04f1-4ceb-bd4b-bdbde7532ed0 None admin] PATH_INFO: `/v3/projects/9bb3822e6d1c4f82815dbeda0f6eff52` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:40.494038 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46490151-04f1-4ceb-bd4b-bdbde7532ed0 None admin] RBAC: Authorizing `identity:get_project(project_id=9bb3822e6d1c4f82815dbeda0f6eff52)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:40.496811 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46490151-04f1-4ceb-bd4b-bdbde7532ed0 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:40.499172 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 185/379] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:40 2022] GET /identity/v3/projects/9bb3822e6d1c4f82815dbeda0f6eff52 => generated 344 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:40.511001 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46490151-04f1-4ceb-bd4b-bdbde7532ed0 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:40.531143 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 195/380] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:40.565488 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a4f473-0c9c-43d9-ba75-9c2c56f26a6c None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:40.674625 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46490151-04f1-4ceb-bd4b-bdbde7532ed0 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:40.706698 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:40.719750 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7793e-19df-42e6-abf1-c84f758b7628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:40.720806 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7793e-19df-42e6-abf1-c84f758b7628 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:40.721413 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7793e-19df-42e6-abf1-c84f758b7628 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:40.721889 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7793e-19df-42e6-abf1-c84f758b7628 None admin] PATH_INFO: `/v3/groups/04484d0cd9c54c72bf4ae174e1cce60e` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:40.728730 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f7793e-19df-42e6-abf1-c84f758b7628 None admin] RBAC: Authorizing `identity:get_group(group_id=04484d0cd9c54c72bf4ae174e1cce60e)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:40.730571 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f7793e-19df-42e6-abf1-c84f758b7628 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:40.737260 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:40.737580 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 186/381] 10.222.0.39 () {56 vars in 1180 bytes} [Fri Aug 5 19:49:40 2022] GET /identity/v3/groups/04484d0cd9c54c72bf4ae174e1cce60e => generated 229 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:40.741019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7793e-19df-42e6-abf1-c84f758b7628 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:40.770992 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a4f473-0c9c-43d9-ba75-9c2c56f26a6c None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:40.816477 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726aeebe-e2e6-4fe3-adcc-e492a6582069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:40.817176 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726aeebe-e2e6-4fe3-adcc-e492a6582069 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:40.817450 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726aeebe-e2e6-4fe3-adcc-e492a6582069 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:40.817749 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726aeebe-e2e6-4fe3-adcc-e492a6582069 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:40.830367 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726aeebe-e2e6-4fe3-adcc-e492a6582069 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:40.835708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-726aeebe-e2e6-4fe3-adcc-e492a6582069 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:40.835708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:40.835708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:40.835708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:40.835708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:40.835708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:40.835708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:40.835708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:40.835708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:40.835708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:40.835708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:40.835708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:40.835708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:40.835708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:40.835708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:40.835708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:40.836666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:40.836666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:40.836666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:40.836666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:40.836666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:40.836666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:40.836666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:40.836666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:40.836666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:40.836666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:40.836666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:40.836666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:40.836666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:40.836666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:40.836666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:40.836666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:40.837502 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:40.837502 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:40.837502 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:40.837502 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:40.837502 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:40.837502 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:40.837502 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:40.837502 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:40.837502 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:40.837502 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:40.837502 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:40.837502 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:49:40.837502 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 196/382] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:40.844085 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726aeebe-e2e6-4fe3-adcc-e492a6582069 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:40.970475 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726aeebe-e2e6-4fe3-adcc-e492a6582069 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:41.015703 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8fafe-8896-4652-bc01-e57451ddfaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:41.016385 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8fafe-8896-4652-bc01-e57451ddfaed None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:41.016701 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8fafe-8896-4652-bc01-e57451ddfaed None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:41.017023 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8fafe-8896-4652-bc01-e57451ddfaed None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:41.017877 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8fafe-8896-4652-bc01-e57451ddfaed None admin] RBAC: Adding query filter params (name=member) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:41.018193 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8fafe-8896-4652-bc01-e57451ddfaed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:41.019799 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8fafe-8896-4652-bc01-e57451ddfaed None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:41.031997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 197/383] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:41.039501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8fafe-8896-4652-bc01-e57451ddfaed None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:41.056259 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7793e-19df-42e6-abf1-c84f758b7628 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:41.066004 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:41.131446 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa4a19-466f-4f38-b3ee-196390fccf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:41.133198 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa4a19-466f-4f38-b3ee-196390fccf45 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:41.133661 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa4a19-466f-4f38-b3ee-196390fccf45 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:41.134135 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa4a19-466f-4f38-b3ee-196390fccf45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:41.135552 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aa4a19-466f-4f38-b3ee-196390fccf45 None admin] RBAC: Adding query filter params (group.id=04484d0cd9c54c72bf4ae174e1cce60e, role.id=89919f784ad144529b07c8ab92ee7d48, scope.project.id=9bb3822e6d1c4f82815dbeda0f6eff52) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:41.140631 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aa4a19-466f-4f38-b3ee-196390fccf45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:41.140631 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aa4a19-466f-4f38-b3ee-196390fccf45 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:41.163587 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 187/384] 10.222.0.39 () {56 vars in 1376 bytes} [Fri Aug 5 19:49:40 2022] GET /identity/v3/role_assignments?group.id=04484d0cd9c54c72bf4ae174e1cce60e&scope.project.id=9bb3822e6d1c4f82815dbeda0f6eff52&role.id=89919f784ad144529b07c8ab92ee7d48 => generated 260 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:41.265514 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8fafe-8896-4652-bc01-e57451ddfaed None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:41.321776 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da9ddd-00b6-44eb-a6bf-cac2d5954971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:41.322837 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da9ddd-00b6-44eb-a6bf-cac2d5954971 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:41.323319 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da9ddd-00b6-44eb-a6bf-cac2d5954971 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:41.323852 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da9ddd-00b6-44eb-a6bf-cac2d5954971 None admin] PATH_INFO: `/v3/groups/f2e4c5e7f5df4798982e58481f941bfe` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:41.325628 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44da9ddd-00b6-44eb-a6bf-cac2d5954971 None admin] RBAC: Authorizing `identity:get_group(group_id=f2e4c5e7f5df4798982e58481f941bfe)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:41.327862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44da9ddd-00b6-44eb-a6bf-cac2d5954971 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:41.330406 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 198/385] 10.222.0.39 () {56 vars in 1180 bytes} [Fri Aug 5 19:49:41 2022] GET /identity/v3/groups/f2e4c5e7f5df4798982e58481f941bfe => generated 226 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:41.335873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa4a19-466f-4f38-b3ee-196390fccf45 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:41.429897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:41.501253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa4a19-466f-4f38-b3ee-196390fccf45 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:41.562595 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a442c-850f-411d-bf0e-3ab500475aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:41.563747 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a442c-850f-411d-bf0e-3ab500475aee None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:41.564245 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a442c-850f-411d-bf0e-3ab500475aee None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:41.564769 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a442c-850f-411d-bf0e-3ab500475aee None admin] PATH_INFO: `/v3/projects/f542d45b34d849d2aad08bc75ebc275a` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:41.569531 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a442c-850f-411d-bf0e-3ab500475aee None admin] RBAC: Authorizing `identity:get_project(project_id=f542d45b34d849d2aad08bc75ebc275a)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:41.571194 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a442c-850f-411d-bf0e-3ab500475aee None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:41.573102 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 188/386] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:41 2022] GET /identity/v3/projects/f542d45b34d849d2aad08bc75ebc275a => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:41.579888 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da9ddd-00b6-44eb-a6bf-cac2d5954971 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:41.703110 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4291ff-0a24-49b2-8b9f-fc809295f093 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:41.703879 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4291ff-0a24-49b2-8b9f-fc809295f093 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:41.704380 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4291ff-0a24-49b2-8b9f-fc809295f093 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:41.706257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 189/387] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:49:41.710736 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:41.712526 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1083f4-7b81-4b76-8617-397fdb2be188 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:41.713063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1083f4-7b81-4b76-8617-397fdb2be188 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:41.713702 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1083f4-7b81-4b76-8617-397fdb2be188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:41.736425 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-5d1083f4-7b81-4b76-8617-397fdb2be188 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:41.738051 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-5d1083f4-7b81-4b76-8617-397fdb2be188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:41.747945 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da9ddd-00b6-44eb-a6bf-cac2d5954971 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:41.817084 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53fbe1-ef9a-4186-87d0-a379330782a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:41.818150 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53fbe1-ef9a-4186-87d0-a379330782a0 None admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:41.818637 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53fbe1-ef9a-4186-87d0-a379330782a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:41.819298 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53fbe1-ef9a-4186-87d0-a379330782a0 None admin] PATH_INFO: `/v3/projects/f542d45b34d849d2aad08bc75ebc275a/groups/f2e4c5e7f5df4798982e58481f941bfe/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:41.826143 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b53fbe1-ef9a-4186-87d0-a379330782a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=f542d45b34d849d2aad08bc75ebc275a, group_id=f2e4c5e7f5df4798982e58481f941bfe, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:41.828918 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b53fbe1-ef9a-4186-87d0-a379330782a0 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:41.852331 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 199/388] 10.222.0.39 () {58 vars in 1442 bytes} [Fri Aug 5 19:49:41 2022] PUT /identity/v3/projects/f542d45b34d849d2aad08bc75ebc275a/groups/f2e4c5e7f5df4798982e58481f941bfe/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:49:41.875181 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 190/389] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:41.898172 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53fbe1-ef9a-4186-87d0-a379330782a0 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:42.043248 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53fbe1-ef9a-4186-87d0-a379330782a0 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:42.093744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05b5db-c088-441d-8ce0-c8b1d51781f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:42.095444 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05b5db-c088-441d-8ce0-c8b1d51781f5 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:42.095984 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05b5db-c088-441d-8ce0-c8b1d51781f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:42.096478 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05b5db-c088-441d-8ce0-c8b1d51781f5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:42.105876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:42.113611 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05b5db-c088-441d-8ce0-c8b1d51781f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:42.120383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-fa05b5db-c088-441d-8ce0-c8b1d51781f5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:42.120383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:42.120383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:42.120383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:42.120383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:42.120383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:42.120383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:42.120383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:42.120383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:42.120383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:42.120383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:42.120383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:42.120383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:42.120383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:42.120383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:42.120383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:42.121506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:42.121506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:42.121506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:42.121506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:42.121506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:42.121506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:42.121506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:42.121506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:42.121506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:42.121506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:42.121506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:42.121506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:42.121506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:42.121506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:42.121506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:42.121506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:42.122374 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:42.122374 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:42.122374 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:42.122374 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:42.122374 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:42.122374 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:42.122374 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:42.122374 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:42.122374 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:42.122374 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:42.122374 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:42.122374 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:49:42.122374 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 200/390] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:42.129263 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1083f4-7b81-4b76-8617-397fdb2be188 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:42.244970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:42.316807 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1083f4-7b81-4b76-8617-397fdb2be188 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:42.360130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6582ac-bb91-4b06-8488-72572f68dfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:42.360749 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6582ac-bb91-4b06-8488-72572f68dfdb None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:42.361036 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6582ac-bb91-4b06-8488-72572f68dfdb None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:42.361316 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6582ac-bb91-4b06-8488-72572f68dfdb None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:42.362124 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6582ac-bb91-4b06-8488-72572f68dfdb None admin] RBAC: Adding query filter params (name=member) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:42.362434 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6582ac-bb91-4b06-8488-72572f68dfdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:42.363831 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6582ac-bb91-4b06-8488-72572f68dfdb None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:42.385063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 191/391] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:42.390516 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05b5db-c088-441d-8ce0-c8b1d51781f5 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:42.452236 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e05513-ae25-44a2-b8aa-71126bc78642 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:42.452829 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e05513-ae25-44a2-b8aa-71126bc78642 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:42.453062 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e05513-ae25-44a2-b8aa-71126bc78642 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:42.454610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 192/392] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:42.461015 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a777dd2-ab09-4b7c-864c-2199e98b7011 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:42.461297 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a777dd2-ab09-4b7c-864c-2199e98b7011 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:42.461574 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a777dd2-ab09-4b7c-864c-2199e98b7011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:42.484098 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-9a777dd2-ab09-4b7c-864c-2199e98b7011 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:42.485628 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-9a777dd2-ab09-4b7c-864c-2199e98b7011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:42.590367 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 193/393] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:49:42.597761 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05b5db-c088-441d-8ce0-c8b1d51781f5 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:42.644365 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2b275-9f8b-4aa1-b551-cac30b19ffd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:42.645046 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2b275-9f8b-4aa1-b551-cac30b19ffd4 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:42.645324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2b275-9f8b-4aa1-b551-cac30b19ffd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:42.645598 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2b275-9f8b-4aa1-b551-cac30b19ffd4 None admin] PATH_INFO: `/v3/groups/f2e4c5e7f5df4798982e58481f941bfe` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:42.646818 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2b275-9f8b-4aa1-b551-cac30b19ffd4 None admin] RBAC: Authorizing `identity:get_group(group_id=f2e4c5e7f5df4798982e58481f941bfe)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:42.648359 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2b275-9f8b-4aa1-b551-cac30b19ffd4 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:42.650848 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 201/394] 10.222.0.39 () {56 vars in 1180 bytes} [Fri Aug 5 19:49:42 2022] GET /identity/v3/groups/f2e4c5e7f5df4798982e58481f941bfe => generated 226 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:42.655770 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a777dd2-ab09-4b7c-864c-2199e98b7011 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:42.665174 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2b275-9f8b-4aa1-b551-cac30b19ffd4 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:42.744328 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:42.788943 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:42.825740 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2b275-9f8b-4aa1-b551-cac30b19ffd4 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:42.879917 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a862166-da3f-4f14-93f0-3e6ea2ed90fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:42.880802 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a862166-da3f-4f14-93f0-3e6ea2ed90fd None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:42.881091 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a862166-da3f-4f14-93f0-3e6ea2ed90fd None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:42.881372 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a862166-da3f-4f14-93f0-3e6ea2ed90fd None admin] PATH_INFO: `/v3/projects/c0ded24d8148467b9a56fa55332feb56` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:42.882021 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a777dd2-ab09-4b7c-864c-2199e98b7011 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:42.882566 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a862166-da3f-4f14-93f0-3e6ea2ed90fd None admin] RBAC: Authorizing `identity:get_project(project_id=c0ded24d8148467b9a56fa55332feb56)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:42.884519 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a862166-da3f-4f14-93f0-3e6ea2ed90fd None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:42.886732 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 202/395] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:42 2022] GET /identity/v3/projects/c0ded24d8148467b9a56fa55332feb56 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:42.892801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a862166-da3f-4f14-93f0-3e6ea2ed90fd None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:42.984621 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d17ad-d8e3-42a3-83aa-56290b658b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:42.985228 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d17ad-d8e3-42a3-83aa-56290b658b07 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:42.985514 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d17ad-d8e3-42a3-83aa-56290b658b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:42.985791 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d17ad-d8e3-42a3-83aa-56290b658b07 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:43.011004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d17ad-d8e3-42a3-83aa-56290b658b07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:43.017508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-ec2d17ad-d8e3-42a3-83aa-56290b658b07 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:43.017508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:43.017508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:43.017508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:43.017508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:43.017508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:43.017508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:43.017508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:43.017508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:43.017508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:43.017508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:43.017508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:43.017508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:43.017508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:43.017508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:43.017508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:43.018405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:43.018405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:43.018405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:43.018405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:43.018405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:43.018405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:43.018405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:43.018405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:43.018405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:43.018405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:43.018405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:43.018405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:43.018405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:43.018405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:43.018405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:43.018405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:43.019211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:43.019211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:43.019211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:43.019211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:43.019211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:43.019211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:43.019211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:43.019211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:43.019211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:43.019211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:43.019211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:43.019211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:43.019211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 194/396] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Aug 5 19:49:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:43.025294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d17ad-d8e3-42a3-83aa-56290b658b07 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:43.124630 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a862166-da3f-4f14-93f0-3e6ea2ed90fd None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:43.148853 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d17ad-d8e3-42a3-83aa-56290b658b07 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:43.150798 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:43.187042 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d1d7d7-09e9-4c5b-8866-0f16fb1698a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:43.187722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1d7d7-09e9-4c5b-8866-0f16fb1698a6 None admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:43.188001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1d7d7-09e9-4c5b-8866-0f16fb1698a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:43.188274 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1d7d7-09e9-4c5b-8866-0f16fb1698a6 None admin] PATH_INFO: `/v3/projects/c0ded24d8148467b9a56fa55332feb56/groups/f2e4c5e7f5df4798982e58481f941bfe/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:43.190366 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d1d7d7-09e9-4c5b-8866-0f16fb1698a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0ded24d8148467b9a56fa55332feb56, group_id=f2e4c5e7f5df4798982e58481f941bfe, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:43.192394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d1d7d7-09e9-4c5b-8866-0f16fb1698a6 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:43.195084 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:43.213416 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37e175-4538-409d-a206-295077957be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:43.214591 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37e175-4538-409d-a206-295077957be6 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:43.214798 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 203/397] 10.222.0.39 () {58 vars in 1442 bytes} [Fri Aug 5 19:49:42 2022] PUT /identity/v3/projects/c0ded24d8148467b9a56fa55332feb56/groups/f2e4c5e7f5df4798982e58481f941bfe/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:49:43.215314 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37e175-4538-409d-a206-295077957be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:43.215820 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37e175-4538-409d-a206-295077957be6 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:43.217057 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b37e175-4538-409d-a206-295077957be6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:43.218037 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b37e175-4538-409d-a206-295077957be6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:43.220294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b37e175-4538-409d-a206-295077957be6 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:43.233485 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 195/398] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:43.240354 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d1d7d7-09e9-4c5b-8866-0f16fb1698a6 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:43.245751 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa666e-28ac-4325-9cb6-b05c5c84beb3 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:43.246438 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa666e-28ac-4325-9cb6-b05c5c84beb3 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:43.246916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa666e-28ac-4325-9cb6-b05c5c84beb3 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:43.249438 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 196/399] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:49:43.254895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513be86-fbd9-472a-b7aa-f07992ac2ae3 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:43.255262 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513be86-fbd9-472a-b7aa-f07992ac2ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:43.255400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513be86-fbd9-472a-b7aa-f07992ac2ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:43.276254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-c513be86-fbd9-472a-b7aa-f07992ac2ae3 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:43.277901 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-c513be86-fbd9-472a-b7aa-f07992ac2ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:43.364707 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 197/400] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:43.385279 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513be86-fbd9-472a-b7aa-f07992ac2ae3 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:43.493674 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d1d7d7-09e9-4c5b-8866-0f16fb1698a6 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:43.494702 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:43.508268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513be86-fbd9-472a-b7aa-f07992ac2ae3 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:43.539283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eecf13-c5ba-4342-9d3d-5e521f50a575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:43.540256 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eecf13-c5ba-4342-9d3d-5e521f50a575 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:43.541018 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eecf13-c5ba-4342-9d3d-5e521f50a575 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:43.541308 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eecf13-c5ba-4342-9d3d-5e521f50a575 None admin] PATH_INFO: `/v3/groups/f2e4c5e7f5df4798982e58481f941bfe` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:43.542986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eecf13-c5ba-4342-9d3d-5e521f50a575 None admin] RBAC: Authorizing `identity:get_group(group_id=f2e4c5e7f5df4798982e58481f941bfe)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:43.545201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eecf13-c5ba-4342-9d3d-5e521f50a575 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:43.547398 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 204/401] 10.222.0.39 () {56 vars in 1180 bytes} [Fri Aug 5 19:49:43 2022] GET /identity/v3/groups/f2e4c5e7f5df4798982e58481f941bfe => generated 226 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:43.553027 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eecf13-c5ba-4342-9d3d-5e521f50a575 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:43.553971 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3516a6-58c5-408a-aa9d-e4be3387bf4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:43.554623 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3516a6-58c5-408a-aa9d-e4be3387bf4f None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:43.554901 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3516a6-58c5-408a-aa9d-e4be3387bf4f None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:43.555173 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3516a6-58c5-408a-aa9d-e4be3387bf4f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:43.570452 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3516a6-58c5-408a-aa9d-e4be3387bf4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:43.576897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-ee3516a6-58c5-408a-aa9d-e4be3387bf4f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:43.576897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:43.576897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:43.576897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:43.576897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:43.576897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:43.576897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:43.576897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:43.576897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:43.576897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:43.576897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:43.576897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:43.576897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:43.576897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:43.576897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:43.576897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:43.577935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:43.577935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:43.577935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:43.577935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:43.577935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:43.577935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:43.577935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:43.577935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:43.577935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:43.577935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:43.577935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:43.577935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:43.577935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:43.577935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:43.577935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:43.577935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:43.578727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:43.578727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:43.578727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:43.578727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:43.578727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:43.578727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:43.578727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:43.578727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:43.578727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:43.578727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:43.578727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:43.578727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:43.578727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 198/402] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Aug 5 19:49:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:43.585338 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3516a6-58c5-408a-aa9d-e4be3387bf4f None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:43.666882 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:43.732295 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eecf13-c5ba-4342-9d3d-5e521f50a575 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:43.775938 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3516a6-58c5-408a-aa9d-e4be3387bf4f None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:43.806555 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e29235-346f-46ee-9673-6705cf10b472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:43.807583 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e29235-346f-46ee-9673-6705cf10b472 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:43.808064 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e29235-346f-46ee-9673-6705cf10b472 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:43.808534 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e29235-346f-46ee-9673-6705cf10b472 None admin] PATH_INFO: `/v3/projects/f542d45b34d849d2aad08bc75ebc275a` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:43.812599 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68547e-455b-450c-b34f-298288c0cf67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:43.812599 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68547e-455b-450c-b34f-298288c0cf67 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:43.812599 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68547e-455b-450c-b34f-298288c0cf67 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:43.812599 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68547e-455b-450c-b34f-298288c0cf67 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:43.812946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d68547e-455b-450c-b34f-298288c0cf67 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:43.812946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d68547e-455b-450c-b34f-298288c0cf67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:43.813957 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d68547e-455b-450c-b34f-298288c0cf67 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:43.814072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e29235-346f-46ee-9673-6705cf10b472 None admin] RBAC: Authorizing `identity:get_project(project_id=f542d45b34d849d2aad08bc75ebc275a)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:43.815709 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e29235-346f-46ee-9673-6705cf10b472 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:43.817706 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 205/403] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:43 2022] GET /identity/v3/projects/f542d45b34d849d2aad08bc75ebc275a => generated 298 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:43.832782 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 199/404] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:43.835242 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e29235-346f-46ee-9673-6705cf10b472 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:43.835355 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68547e-455b-450c-b34f-298288c0cf67 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:43.855815 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:43.951278 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68547e-455b-450c-b34f-298288c0cf67 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:43.988779 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2144ed0f-2c1c-47c3-9588-bdfb8d80257a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:43.989687 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144ed0f-2c1c-47c3-9588-bdfb8d80257a None admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:43.990102 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144ed0f-2c1c-47c3-9588-bdfb8d80257a None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:43.990539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144ed0f-2c1c-47c3-9588-bdfb8d80257a None admin] PATH_INFO: `/v3/projects/f542d45b34d849d2aad08bc75ebc275a/groups/f2e4c5e7f5df4798982e58481f941bfe/roles/0eafd8a7ba2d47faaf5aecbb3360143b` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:43.992893 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2144ed0f-2c1c-47c3-9588-bdfb8d80257a None admin] RBAC: Authorizing `identity:create_grant(project_id=f542d45b34d849d2aad08bc75ebc275a, group_id=f2e4c5e7f5df4798982e58481f941bfe, role_id=0eafd8a7ba2d47faaf5aecbb3360143b)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:43.994920 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2144ed0f-2c1c-47c3-9588-bdfb8d80257a None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:44.004647 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e29235-346f-46ee-9673-6705cf10b472 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:44.021544 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 200/405] 10.222.0.39 () {58 vars in 1442 bytes} [Fri Aug 5 19:49:43 2022] PUT /identity/v3/projects/f542d45b34d849d2aad08bc75ebc275a/groups/f2e4c5e7f5df4798982e58481f941bfe/roles/0eafd8a7ba2d47faaf5aecbb3360143b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:49:44.041076 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88adfd42-f4ed-4b23-a48b-d1f7348c4745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:44.041719 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88adfd42-f4ed-4b23-a48b-d1f7348c4745 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:44.042079 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88adfd42-f4ed-4b23-a48b-d1f7348c4745 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:44.042442 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88adfd42-f4ed-4b23-a48b-d1f7348c4745 None admin] PATH_INFO: `/v3/groups/f2e4c5e7f5df4798982e58481f941bfe` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:44.043795 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88adfd42-f4ed-4b23-a48b-d1f7348c4745 None admin] RBAC: Authorizing `identity:get_group(group_id=f2e4c5e7f5df4798982e58481f941bfe)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:44.045971 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88adfd42-f4ed-4b23-a48b-d1f7348c4745 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:44.048237 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 206/406] 10.222.0.39 () {56 vars in 1180 bytes} [Fri Aug 5 19:49:43 2022] GET /identity/v3/groups/f2e4c5e7f5df4798982e58481f941bfe => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:44.048742 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:44.054572 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2144ed0f-2c1c-47c3-9588-bdfb8d80257a None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:44.067160 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf799f3-5767-4e3f-9602-4a6fa0184a96 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:44.067511 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf799f3-5767-4e3f-9602-4a6fa0184a96 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:44.067727 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf799f3-5767-4e3f-9602-4a6fa0184a96 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:44.069163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 207/407] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:44.075040 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59096b71-ab4e-46c8-9664-d85b695f142c None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:44.075474 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59096b71-ab4e-46c8-9664-d85b695f142c None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:44.075854 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59096b71-ab4e-46c8-9664-d85b695f142c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:44.095749 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-59096b71-ab4e-46c8-9664-d85b695f142c None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:44.097280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-59096b71-ab4e-46c8-9664-d85b695f142c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:44.177668 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 208/408] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:49:44.214371 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59096b71-ab4e-46c8-9664-d85b695f142c None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:44.230705 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:44.258796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2144ed0f-2c1c-47c3-9588-bdfb8d80257a None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:44.300498 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a67c2e-5ca5-4f62-88f7-fe3931cc965f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:44.301330 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a67c2e-5ca5-4f62-88f7-fe3931cc965f None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:44.301754 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a67c2e-5ca5-4f62-88f7-fe3931cc965f None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:44.302133 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a67c2e-5ca5-4f62-88f7-fe3931cc965f None admin] PATH_INFO: `/v3/projects/c0ded24d8148467b9a56fa55332feb56` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:44.303742 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a67c2e-5ca5-4f62-88f7-fe3931cc965f None admin] RBAC: Authorizing `identity:get_project(project_id=c0ded24d8148467b9a56fa55332feb56)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:44.305347 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a67c2e-5ca5-4f62-88f7-fe3931cc965f None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:44.307760 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 201/409] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:44 2022] GET /identity/v3/projects/c0ded24d8148467b9a56fa55332feb56 => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:44.314958 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a67c2e-5ca5-4f62-88f7-fe3931cc965f None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:44.358232 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59096b71-ab4e-46c8-9664-d85b695f142c None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:44.414185 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:44.427962 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9a4c4-1cf3-473a-a048-15f7499ce569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:44.429379 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9a4c4-1cf3-473a-a048-15f7499ce569 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:44.429961 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9a4c4-1cf3-473a-a048-15f7499ce569 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:44.431017 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9a4c4-1cf3-473a-a048-15f7499ce569 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:44.446282 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d9a4c4-1cf3-473a-a048-15f7499ce569 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:44.453441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-b5d9a4c4-1cf3-473a-a048-15f7499ce569 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:49:44.453441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:44.453441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:44.453441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:44.453441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:44.453441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:44.453441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:44.453441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:44.453441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:44.453441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:44.453441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:44.453441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:44.453441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:44.453441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:44.453441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:44.453441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:44.454522 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:44.454522 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:44.454522 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:44.454522 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:44.454522 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:44.454522 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:44.454522 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:44.454522 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:44.454522 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:44.454522 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:44.454522 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:44.454522 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:44.454522 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:44.454522 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:44.454522 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:44.454522 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:44.455490 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:44.455490 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:44.455490 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:44.455490 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:44.455490 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:44.455490 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:44.455490 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:44.455490 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:44.455490 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:44.455490 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:44.455490 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:49:44.455490 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:49:44.456219 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 209/410] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:49:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:44.460601 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521fa40-8c36-43fd-9150-478e2f8e18e7 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:44.460601 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521fa40-8c36-43fd-9150-478e2f8e18e7 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:44.460601 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521fa40-8c36-43fd-9150-478e2f8e18e7 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:44.460601 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 210/411] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:44.467497 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521fa40-8c36-43fd-9150-478e2f8e18e7 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:44.495206 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a67c2e-5ca5-4f62-88f7-fe3931cc965f None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:44.544476 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696af2da-b7d0-4131-9c5b-d4efe5b74a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:44.546773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696af2da-b7d0-4131-9c5b-d4efe5b74a57 None admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:44.546773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696af2da-b7d0-4131-9c5b-d4efe5b74a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:44.546773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696af2da-b7d0-4131-9c5b-d4efe5b74a57 None admin] PATH_INFO: `/v3/projects/c0ded24d8148467b9a56fa55332feb56/groups/f2e4c5e7f5df4798982e58481f941bfe/roles/0eafd8a7ba2d47faaf5aecbb3360143b` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:44.550748 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696af2da-b7d0-4131-9c5b-d4efe5b74a57 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0ded24d8148467b9a56fa55332feb56, group_id=f2e4c5e7f5df4798982e58481f941bfe, role_id=0eafd8a7ba2d47faaf5aecbb3360143b)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:44.552315 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696af2da-b7d0-4131-9c5b-d4efe5b74a57 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:44.570964 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 202/412] 10.222.0.39 () {58 vars in 1442 bytes} [Fri Aug 5 19:49:44 2022] PUT /identity/v3/projects/c0ded24d8148467b9a56fa55332feb56/groups/f2e4c5e7f5df4798982e58481f941bfe/roles/0eafd8a7ba2d47faaf5aecbb3360143b => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:44.573098 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f97eb0-aa68-487b-939d-011883e9097e None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:44.573784 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f97eb0-aa68-487b-939d-011883e9097e None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:44.574172 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f97eb0-aa68-487b-939d-011883e9097e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:44.584669 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:44.591686 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521fa40-8c36-43fd-9150-478e2f8e18e7 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:44.603456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-76f97eb0-aa68-487b-939d-011883e9097e None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:44.605749 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-76f97eb0-aa68-487b-939d-011883e9097e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:44.651870 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c771ec3-586a-4089-855b-c87d174fd71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:44.652657 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c771ec3-586a-4089-855b-c87d174fd71d None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:44.653010 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c771ec3-586a-4089-855b-c87d174fd71d None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:44.653583 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c771ec3-586a-4089-855b-c87d174fd71d None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:44.654508 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c771ec3-586a-4089-855b-c87d174fd71d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:44.655740 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c771ec3-586a-4089-855b-c87d174fd71d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:44.657280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c771ec3-586a-4089-855b-c87d174fd71d None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:44.676145 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 211/413] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:44.682096 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c771ec3-586a-4089-855b-c87d174fd71d None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:44.768198 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 203/414] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:44.781591 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:44.791458 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f97eb0-aa68-487b-939d-011883e9097e None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:44.845295 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c771ec3-586a-4089-855b-c87d174fd71d None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:44.891069 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587fbf8-b99e-4b36-b55d-f83fd973d784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:44.891723 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587fbf8-b99e-4b36-b55d-f83fd973d784 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:44.892004 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587fbf8-b99e-4b36-b55d-f83fd973d784 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:44.892275 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587fbf8-b99e-4b36-b55d-f83fd973d784 None admin] PATH_INFO: `/v3/groups/04484d0cd9c54c72bf4ae174e1cce60e` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:44.893538 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f587fbf8-b99e-4b36-b55d-f83fd973d784 None admin] RBAC: Authorizing `identity:get_group(group_id=04484d0cd9c54c72bf4ae174e1cce60e)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:44.894971 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f587fbf8-b99e-4b36-b55d-f83fd973d784 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:44.900761 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 212/415] 10.222.0.39 () {56 vars in 1180 bytes} [Fri Aug 5 19:49:44 2022] GET /identity/v3/groups/04484d0cd9c54c72bf4ae174e1cce60e => generated 229 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:44.913362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587fbf8-b99e-4b36-b55d-f83fd973d784 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:44.919995 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f97eb0-aa68-487b-939d-011883e9097e None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:44.958052 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22b1c1-45ff-4d75-b458-856fe9e6507e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:44.958697 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22b1c1-45ff-4d75-b458-856fe9e6507e None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:44.958977 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22b1c1-45ff-4d75-b458-856fe9e6507e None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:44.959249 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22b1c1-45ff-4d75-b458-856fe9e6507e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:44.961193 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:44.970869 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa22b1c1-45ff-4d75-b458-856fe9e6507e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:44.976063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-aa22b1c1-45ff-4d75-b458-856fe9e6507e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:44.976063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:44.976063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:44.976063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:44.976063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:44.976063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:44.976063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:44.976063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:44.976063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:44.976063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:44.976063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:44.976063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:44.976063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:44.976063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:44.976063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:44.976063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:44.977039 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:44.977039 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:44.977039 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:44.977039 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:44.977039 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:44.977039 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:44.977039 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:44.977039 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:44.977039 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:44.977039 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:44.977039 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:44.977039 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:44.977039 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:44.977039 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:44.977039 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:44.977039 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:44.978263 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:44.978263 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:44.978263 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:44.978263 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:44.978263 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:44.978263 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:44.978263 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:44.978263 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:44.978263 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:44.978263 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:44.978263 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:44.978263 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:44.978263 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 204/416] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:44.988256 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22b1c1-45ff-4d75-b458-856fe9e6507e None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:45.098137 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587fbf8-b99e-4b36-b55d-f83fd973d784 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:45.125452 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:45.160714 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790cb905-1e65-41e6-b918-3e91bafbfa38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:45.161363 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cb905-1e65-41e6-b918-3e91bafbfa38 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:45.161643 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cb905-1e65-41e6-b918-3e91bafbfa38 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:45.161919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cb905-1e65-41e6-b918-3e91bafbfa38 None admin] PATH_INFO: `/v3/projects/9bb3822e6d1c4f82815dbeda0f6eff52` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:45.165338 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790cb905-1e65-41e6-b918-3e91bafbfa38 None admin] RBAC: Authorizing `identity:get_project(project_id=9bb3822e6d1c4f82815dbeda0f6eff52)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:45.166795 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790cb905-1e65-41e6-b918-3e91bafbfa38 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:45.168474 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 213/417] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:44 2022] GET /identity/v3/projects/9bb3822e6d1c4f82815dbeda0f6eff52 => generated 344 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:45.183105 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790cb905-1e65-41e6-b918-3e91bafbfa38 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:45.214134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22b1c1-45ff-4d75-b458-856fe9e6507e None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:45.284275 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce46e5ca-e719-4d47-b8b6-18d360f70f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:45.285278 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce46e5ca-e719-4d47-b8b6-18d360f70f0e None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:45.285767 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce46e5ca-e719-4d47-b8b6-18d360f70f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:45.286246 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce46e5ca-e719-4d47-b8b6-18d360f70f0e None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:45.287192 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce46e5ca-e719-4d47-b8b6-18d360f70f0e None admin] RBAC: Adding query filter params (name=member) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:45.287685 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce46e5ca-e719-4d47-b8b6-18d360f70f0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:45.289382 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce46e5ca-e719-4d47-b8b6-18d360f70f0e None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:45.308062 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 205/418] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:45.313340 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce46e5ca-e719-4d47-b8b6-18d360f70f0e None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:45.341723 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790cb905-1e65-41e6-b918-3e91bafbfa38 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:45.417354 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1175c9-f942-48a0-8a29-d45998ebb631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:45.418482 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1175c9-f942-48a0-8a29-d45998ebb631 None admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:45.418982 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1175c9-f942-48a0-8a29-d45998ebb631 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:45.419444 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1175c9-f942-48a0-8a29-d45998ebb631 None admin] PATH_INFO: `/v3/projects/9bb3822e6d1c4f82815dbeda0f6eff52/groups/04484d0cd9c54c72bf4ae174e1cce60e/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:45.421880 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1175c9-f942-48a0-8a29-d45998ebb631 None admin] RBAC: Authorizing `identity:create_grant(project_id=9bb3822e6d1c4f82815dbeda0f6eff52, group_id=04484d0cd9c54c72bf4ae174e1cce60e, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:45.424748 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1175c9-f942-48a0-8a29-d45998ebb631 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:45.427339 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:45.451375 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 214/419] 10.222.0.39 () {58 vars in 1442 bytes} [Fri Aug 5 19:49:45 2022] PUT /identity/v3/projects/9bb3822e6d1c4f82815dbeda0f6eff52/groups/04484d0cd9c54c72bf4ae174e1cce60e/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:49:45.491570 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce46e5ca-e719-4d47-b8b6-18d360f70f0e None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:45.530393 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68489c7-f88a-435f-a7cf-b81fda971399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:45.531635 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68489c7-f88a-435f-a7cf-b81fda971399 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:45.531635 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68489c7-f88a-435f-a7cf-b81fda971399 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:45.531635 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68489c7-f88a-435f-a7cf-b81fda971399 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:45.625325 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68489c7-f88a-435f-a7cf-b81fda971399 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:45.627256 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68489c7-f88a-435f-a7cf-b81fda971399 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:45.702695 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:45.755170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 206/420] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:45.760418 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1175c9-f942-48a0-8a29-d45998ebb631 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:45.816904 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e40201-c4ee-4f95-832c-0270d9cdf071 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:45.817553 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:45.818668 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e40201-c4ee-4f95-832c-0270d9cdf071 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:45.819146 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e40201-c4ee-4f95-832c-0270d9cdf071 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:45.821336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 207/421] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:45.830321 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c7b0c-d78d-4b49-94cd-6aaad8b4bbb9 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:45.833918 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c7b0c-d78d-4b49-94cd-6aaad8b4bbb9 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:45.835360 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c7b0c-d78d-4b49-94cd-6aaad8b4bbb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:45.878423 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-098c7b0c-d78d-4b49-94cd-6aaad8b4bbb9 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:45.880840 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-098c7b0c-d78d-4b49-94cd-6aaad8b4bbb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:45.898857 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1175c9-f942-48a0-8a29-d45998ebb631 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:45.954545 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79231b-b007-4b3d-a3d6-0712cd4bf49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:45.955087 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79231b-b007-4b3d-a3d6-0712cd4bf49d None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:45.955369 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79231b-b007-4b3d-a3d6-0712cd4bf49d None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:45.955632 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79231b-b007-4b3d-a3d6-0712cd4bf49d None admin] PATH_INFO: `/v3/projects/f542d45b34d849d2aad08bc75ebc275a` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:45.956850 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79231b-b007-4b3d-a3d6-0712cd4bf49d None admin] RBAC: Authorizing `identity:get_project(project_id=f542d45b34d849d2aad08bc75ebc275a)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:45.958300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79231b-b007-4b3d-a3d6-0712cd4bf49d None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:45.960318 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 215/422] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:45 2022] GET /identity/v3/projects/f542d45b34d849d2aad08bc75ebc275a => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:45.965897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79231b-b007-4b3d-a3d6-0712cd4bf49d None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:46.016992 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 208/423] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:46.079763 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c7b0c-d78d-4b49-94cd-6aaad8b4bbb9 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:46.114331 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79231b-b007-4b3d-a3d6-0712cd4bf49d None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:46.137128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:46.150411 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5de318-aef7-4ad4-951e-fb20612216ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:46.151063 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5de318-aef7-4ad4-951e-fb20612216ca None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:46.151345 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5de318-aef7-4ad4-951e-fb20612216ca None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:46.151624 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5de318-aef7-4ad4-951e-fb20612216ca None admin] PATH_INFO: `/v3/groups/f2e4c5e7f5df4798982e58481f941bfe` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:46.161780 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5de318-aef7-4ad4-951e-fb20612216ca None admin] RBAC: Authorizing `identity:get_group(group_id=f2e4c5e7f5df4798982e58481f941bfe)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:46.163273 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5de318-aef7-4ad4-951e-fb20612216ca None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:46.165386 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 216/424] 10.222.0.39 () {56 vars in 1180 bytes} [Fri Aug 5 19:49:45 2022] GET /identity/v3/groups/f2e4c5e7f5df4798982e58481f941bfe => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:46.187696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5de318-aef7-4ad4-951e-fb20612216ca None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:46.249292 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:46.273402 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c7b0c-d78d-4b49-94cd-6aaad8b4bbb9 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:46.321286 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5706b98a-c333-4f92-96d3-1862d23b8d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:46.322314 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706b98a-c333-4f92-96d3-1862d23b8d36 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:46.322864 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706b98a-c333-4f92-96d3-1862d23b8d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:46.323143 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706b98a-c333-4f92-96d3-1862d23b8d36 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:46.337611 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5706b98a-c333-4f92-96d3-1862d23b8d36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:46.343267 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5de318-aef7-4ad4-951e-fb20612216ca None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:46.343636 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-5706b98a-c333-4f92-96d3-1862d23b8d36 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:46.343636 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:46.343636 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:46.343636 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:46.343636 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:46.343636 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:46.343636 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:46.343636 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:46.343636 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:46.343636 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:46.343636 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:46.343636 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:46.343636 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:46.343636 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:46.343636 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:46.343636 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:46.344558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:46.344558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:46.344558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:46.344558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:46.344558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:46.344558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:46.344558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:46.344558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:46.344558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:46.344558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:46.344558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:46.344558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:46.344558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:46.344558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:46.344558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:46.344558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:46.345376 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:46.345376 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:46.345376 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:46.345376 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:46.345376 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:46.345376 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:46.345376 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:46.345376 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:46.345376 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:46.345376 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:46.345376 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:46.345376 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:46.345376 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 209/425] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:46.352494 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5706b98a-c333-4f92-96d3-1862d23b8d36 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:46.407436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f3e06-bc64-4de4-b1a4-0594a2f2aed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:46.408971 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f3e06-bc64-4de4-b1a4-0594a2f2aed5 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:46.409252 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f3e06-bc64-4de4-b1a4-0594a2f2aed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:46.409522 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f3e06-bc64-4de4-b1a4-0594a2f2aed5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:46.410314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f3e06-bc64-4de4-b1a4-0594a2f2aed5 None admin] RBAC: Adding query filter params (group.id=f2e4c5e7f5df4798982e58481f941bfe, role.id=fbd7c81b4fba4b6d9e61b1e9bb3e2046, scope.project.id=f542d45b34d849d2aad08bc75ebc275a) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:46.410618 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f3e06-bc64-4de4-b1a4-0594a2f2aed5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:46.414116 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f3e06-bc64-4de4-b1a4-0594a2f2aed5 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:46.430889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 217/426] 10.222.0.39 () {56 vars in 1376 bytes} [Fri Aug 5 19:49:46 2022] GET /identity/v3/role_assignments?group.id=f2e4c5e7f5df4798982e58481f941bfe&scope.project.id=f542d45b34d849d2aad08bc75ebc275a&role.id=fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 611 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:46.521949 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5706b98a-c333-4f92-96d3-1862d23b8d36 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:46.571673 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f1002-b495-4eb7-a740-03f261d3480a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:46.572433 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f1002-b495-4eb7-a740-03f261d3480a None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:46.572663 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f1002-b495-4eb7-a740-03f261d3480a None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:46.572939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f1002-b495-4eb7-a740-03f261d3480a None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:46.573670 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f1002-b495-4eb7-a740-03f261d3480a None admin] RBAC: Adding query filter params (name=member) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:46.573973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f1002-b495-4eb7-a740-03f261d3480a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:46.575372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f1002-b495-4eb7-a740-03f261d3480a None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:46.586368 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 210/427] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:46.594222 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f3e06-bc64-4de4-b1a4-0594a2f2aed5 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:46.609378 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9260e-5caf-48bf-954b-20471e923983 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:46.609378 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9260e-5caf-48bf-954b-20471e923983 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:46.609378 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9260e-5caf-48bf-954b-20471e923983 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:46.613537 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 211/428] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:49:46.617257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51d7ac-affa-41ef-946e-99d3ae50e247 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:46.617535 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51d7ac-affa-41ef-946e-99d3ae50e247 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:46.617807 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51d7ac-affa-41ef-946e-99d3ae50e247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:46.638116 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-7b51d7ac-affa-41ef-946e-99d3ae50e247 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:46.639916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-7b51d7ac-affa-41ef-946e-99d3ae50e247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:46.642837 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:46.677824 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:46.756767 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 212/429] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:46.764798 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f3e06-bc64-4de4-b1a4-0594a2f2aed5 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:46.777125 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51d7ac-affa-41ef-946e-99d3ae50e247 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:46.808681 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568dc75-04e4-4578-acd8-46cc36071f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:46.809647 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568dc75-04e4-4578-acd8-46cc36071f58 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:46.810264 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568dc75-04e4-4578-acd8-46cc36071f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:46.811091 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568dc75-04e4-4578-acd8-46cc36071f58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:46.893687 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9568dc75-04e4-4578-acd8-46cc36071f58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:46.895461 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9568dc75-04e4-4578-acd8-46cc36071f58 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:46.902874 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51d7ac-affa-41ef-946e-99d3ae50e247 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:46.943681 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79191748-c2cc-49d1-853e-b4570d50ff8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:46.944377 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79191748-c2cc-49d1-853e-b4570d50ff8d None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:46.944799 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79191748-c2cc-49d1-853e-b4570d50ff8d None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:46.945065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79191748-c2cc-49d1-853e-b4570d50ff8d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:46.946374 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:46.955497 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:46.961602 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79191748-c2cc-49d1-853e-b4570d50ff8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:46.967606 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-79191748-c2cc-49d1-853e-b4570d50ff8d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:46.967606 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:46.967606 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:46.967606 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:46.967606 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:46.967606 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:46.967606 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:46.967606 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:46.967606 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:46.967606 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:46.967606 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:46.967606 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:46.967606 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:46.967606 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:46.967606 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:46.967606 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:46.969162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:46.969162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:46.969162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:46.969162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:46.969162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:46.969162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:46.969162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:46.969162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:46.969162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:46.969162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:46.969162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:46.969162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:46.969162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:46.969162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:46.969162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:46.969162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:46.970156 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:46.970156 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:46.970156 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:46.970156 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:46.970156 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:46.970156 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:46.970156 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:46.970156 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:46.970156 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:46.970156 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:46.970156 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:46.970156 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:46.976706 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 213/430] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Aug 5 19:49:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:46.979307 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79191748-c2cc-49d1-853e-b4570d50ff8d None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:47.017248 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 218/431] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:47.023279 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568dc75-04e4-4578-acd8-46cc36071f58 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:47.122233 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79191748-c2cc-49d1-853e-b4570d50ff8d None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:47.173034 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568dc75-04e4-4578-acd8-46cc36071f58 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:47.175496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8643e8-140c-4291-93d2-93092212c030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:47.176202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8643e8-140c-4291-93d2-93092212c030 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:47.176598 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8643e8-140c-4291-93d2-93092212c030 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:47.176948 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8643e8-140c-4291-93d2-93092212c030 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:47.177814 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8643e8-140c-4291-93d2-93092212c030 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:47.178227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8643e8-140c-4291-93d2-93092212c030 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:47.179725 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8643e8-140c-4291-93d2-93092212c030 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:47.192697 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 214/432] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:47.196869 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8643e8-140c-4291-93d2-93092212c030 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:47.221094 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4cd6d6-de85-4a34-a1ba-f8026c198c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:47.222051 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4cd6d6-de85-4a34-a1ba-f8026c198c85 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:47.222339 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4cd6d6-de85-4a34-a1ba-f8026c198c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:47.222619 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4cd6d6-de85-4a34-a1ba-f8026c198c85 None admin] PATH_INFO: `/v3/projects/c0ded24d8148467b9a56fa55332feb56` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:47.224461 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4cd6d6-de85-4a34-a1ba-f8026c198c85 None admin] RBAC: Authorizing `identity:get_project(project_id=c0ded24d8148467b9a56fa55332feb56)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:47.225951 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4cd6d6-de85-4a34-a1ba-f8026c198c85 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:47.228956 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 219/433] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:47 2022] GET /identity/v3/projects/c0ded24d8148467b9a56fa55332feb56 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:47.237497 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4cd6d6-de85-4a34-a1ba-f8026c198c85 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:47.261360 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:47.294098 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:47.358965 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8643e8-140c-4291-93d2-93092212c030 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:47.373636 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4cd6d6-de85-4a34-a1ba-f8026c198c85 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:47.412751 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069b742-90cf-46c0-95f2-92f1a85a2c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:47.413401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069b742-90cf-46c0-95f2-92f1a85a2c7e None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:47.413680 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069b742-90cf-46c0-95f2-92f1a85a2c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:47.413957 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069b742-90cf-46c0-95f2-92f1a85a2c7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:47.424115 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676b12f1-135c-4170-8b9a-39b7569763a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:47.424785 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b12f1-135c-4170-8b9a-39b7569763a8 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:47.425059 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b12f1-135c-4170-8b9a-39b7569763a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:47.425334 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b12f1-135c-4170-8b9a-39b7569763a8 None admin] PATH_INFO: `/v3/groups/f2e4c5e7f5df4798982e58481f941bfe` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:47.426479 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676b12f1-135c-4170-8b9a-39b7569763a8 None admin] RBAC: Authorizing `identity:get_group(group_id=f2e4c5e7f5df4798982e58481f941bfe)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:47.427919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676b12f1-135c-4170-8b9a-39b7569763a8 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:47.429792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 220/434] 10.222.0.39 () {56 vars in 1180 bytes} [Fri Aug 5 19:49:47 2022] GET /identity/v3/groups/f2e4c5e7f5df4798982e58481f941bfe => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:47.430780 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b5f8f-d806-4661-8225-4818013ce32e None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:47.431062 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b5f8f-d806-4661-8225-4818013ce32e None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:47.431330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b5f8f-d806-4661-8225-4818013ce32e None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:47.432503 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 221/435] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:47.435202 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b5f8f-d806-4661-8225-4818013ce32e None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:47.500613 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3069b742-90cf-46c0-95f2-92f1a85a2c7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:47.502102 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3069b742-90cf-46c0-95f2-92f1a85a2c7e None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:47.530315 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:47.555157 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b5f8f-d806-4661-8225-4818013ce32e None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:47.579317 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:47.600300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885965cb-fb05-4105-a570-faa82b1492d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:47.600950 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885965cb-fb05-4105-a570-faa82b1492d7 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:47.601229 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885965cb-fb05-4105-a570-faa82b1492d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:47.601561 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885965cb-fb05-4105-a570-faa82b1492d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:47.602342 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885965cb-fb05-4105-a570-faa82b1492d7 None admin] RBAC: Adding query filter params (group.id=f2e4c5e7f5df4798982e58481f941bfe, role.id=fbd7c81b4fba4b6d9e61b1e9bb3e2046, scope.project.id=c0ded24d8148467b9a56fa55332feb56) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:47.602649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885965cb-fb05-4105-a570-faa82b1492d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:47.604072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885965cb-fb05-4105-a570-faa82b1492d7 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:47.611172 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 222/436] 10.222.0.39 () {56 vars in 1376 bytes} [Fri Aug 5 19:49:47 2022] GET /identity/v3/role_assignments?group.id=f2e4c5e7f5df4798982e58481f941bfe&scope.project.id=c0ded24d8148467b9a56fa55332feb56&role.id=fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:47.612212 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ce58f-6e79-4b25-87e5-e9381ee2d096 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:47.612502 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ce58f-6e79-4b25-87e5-e9381ee2d096 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:47.612827 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ce58f-6e79-4b25-87e5-e9381ee2d096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:47.634187 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-ed8ce58f-6e79-4b25-87e5-e9381ee2d096 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:47.635441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-ed8ce58f-6e79-4b25-87e5-e9381ee2d096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:47.654491 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 215/437] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 19:49:47.660653 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069b742-90cf-46c0-95f2-92f1a85a2c7e None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:47.721660 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 223/438] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:47.745399 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ce58f-6e79-4b25-87e5-e9381ee2d096 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:47.799541 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069b742-90cf-46c0-95f2-92f1a85a2c7e None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:47.852509 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ad8fd-1549-475d-b00c-81506e4a1a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:47.853179 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ad8fd-1549-475d-b00c-81506e4a1a66 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:47.853569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ad8fd-1549-475d-b00c-81506e4a1a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:47.853842 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ad8fd-1549-475d-b00c-81506e4a1a66 None admin] PATH_INFO: `/v3/projects/f542d45b34d849d2aad08bc75ebc275a` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:47.855130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ad8fd-1549-475d-b00c-81506e4a1a66 None admin] RBAC: Authorizing `identity:get_project(project_id=f542d45b34d849d2aad08bc75ebc275a)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:47.856544 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ad8fd-1549-475d-b00c-81506e4a1a66 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:47.860615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 216/439] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:47 2022] GET /identity/v3/projects/f542d45b34d849d2aad08bc75ebc275a => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:47.863425 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:47.864764 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ad8fd-1549-475d-b00c-81506e4a1a66 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:47.893981 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ce58f-6e79-4b25-87e5-e9381ee2d096 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:47.896976 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:47.938486 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ee5ff-7416-40f6-9b27-32a2d0132286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:47.939747 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ee5ff-7416-40f6-9b27-32a2d0132286 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:47.940392 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ee5ff-7416-40f6-9b27-32a2d0132286 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:47.940951 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ee5ff-7416-40f6-9b27-32a2d0132286 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:47.958469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ee5ff-7416-40f6-9b27-32a2d0132286 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:47.967364 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-9b6ee5ff-7416-40f6-9b27-32a2d0132286 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:47.967364 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:47.967364 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:47.967364 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:47.967364 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:47.967364 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:47.967364 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:47.967364 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:47.967364 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:47.967364 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:47.967364 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:47.967364 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:47.967364 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:47.967364 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:47.967364 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:47.967364 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:47.968320 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:47.968320 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:47.968320 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:47.968320 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:47.968320 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:47.968320 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:47.968320 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:47.968320 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:47.968320 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:47.968320 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:47.968320 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:47.968320 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:47.968320 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:47.968320 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:47.968320 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:47.968320 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:47.969477 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:47.969477 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:47.969477 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:47.969477 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:47.969477 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:47.969477 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:47.969477 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:47.969477 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:47.969477 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:47.969477 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:47.969477 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:47.969477 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:49:47.970481 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 224/440] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Aug 5 19:49:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:47.975491 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ee5ff-7416-40f6-9b27-32a2d0132286 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:48.000134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ad8fd-1549-475d-b00c-81506e4a1a66 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:48.038628 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904e9da-310f-45e6-83e0-4532a735700c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:48.039256 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904e9da-310f-45e6-83e0-4532a735700c None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:48.039537 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904e9da-310f-45e6-83e0-4532a735700c None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:48.040077 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904e9da-310f-45e6-83e0-4532a735700c None admin] PATH_INFO: `/v3/groups/f2e4c5e7f5df4798982e58481f941bfe` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:48.041701 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904e9da-310f-45e6-83e0-4532a735700c None admin] RBAC: Authorizing `identity:get_group(group_id=f2e4c5e7f5df4798982e58481f941bfe)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:48.043371 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904e9da-310f-45e6-83e0-4532a735700c None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:48.045363 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 217/441] 10.222.0.39 () {56 vars in 1180 bytes} [Fri Aug 5 19:49:47 2022] GET /identity/v3/groups/f2e4c5e7f5df4798982e58481f941bfe => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:48.051203 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904e9da-310f-45e6-83e0-4532a735700c None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:48.097722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ee5ff-7416-40f6-9b27-32a2d0132286 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:48.153819 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a74cc-e5fa-4dd7-ba8c-97a01d9f9761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:48.155260 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a74cc-e5fa-4dd7-ba8c-97a01d9f9761 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:48.155260 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a74cc-e5fa-4dd7-ba8c-97a01d9f9761 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:48.155260 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a74cc-e5fa-4dd7-ba8c-97a01d9f9761 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:48.156263 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77a74cc-e5fa-4dd7-ba8c-97a01d9f9761 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:48.156412 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77a74cc-e5fa-4dd7-ba8c-97a01d9f9761 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:48.157801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77a74cc-e5fa-4dd7-ba8c-97a01d9f9761 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:48.161032 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:48.167697 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 225/442] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:48.169241 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00bbb2-9433-4fa7-ba7e-a90f8d67c1fe None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:48.169713 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00bbb2-9433-4fa7-ba7e-a90f8d67c1fe None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:48.170008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00bbb2-9433-4fa7-ba7e-a90f8d67c1fe None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:48.171271 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 226/443] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:48.173347 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00bbb2-9433-4fa7-ba7e-a90f8d67c1fe None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:48.181959 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:48.189621 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904e9da-310f-45e6-83e0-4532a735700c None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:48.226814 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e5c886-3566-41ca-a04b-b2848fc5138b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:48.227417 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5c886-3566-41ca-a04b-b2848fc5138b None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:48.227694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5c886-3566-41ca-a04b-b2848fc5138b None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:48.227969 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5c886-3566-41ca-a04b-b2848fc5138b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:48.228789 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e5c886-3566-41ca-a04b-b2848fc5138b None admin] RBAC: Adding query filter params (group.id=f2e4c5e7f5df4798982e58481f941bfe, role.id=0eafd8a7ba2d47faaf5aecbb3360143b, scope.project.id=f542d45b34d849d2aad08bc75ebc275a) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:48.229098 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e5c886-3566-41ca-a04b-b2848fc5138b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:48.230537 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e5c886-3566-41ca-a04b-b2848fc5138b None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:48.237015 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 218/444] 10.222.0.39 () {56 vars in 1376 bytes} [Fri Aug 5 19:49:48 2022] GET /identity/v3/role_assignments?group.id=f2e4c5e7f5df4798982e58481f941bfe&scope.project.id=f542d45b34d849d2aad08bc75ebc275a&role.id=0eafd8a7ba2d47faaf5aecbb3360143b => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:48.238184 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690fb36-eef8-4885-a97c-61cb7c5565ad None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:48.238467 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690fb36-eef8-4885-a97c-61cb7c5565ad None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:48.238829 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690fb36-eef8-4885-a97c-61cb7c5565ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:48.259931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-3690fb36-eef8-4885-a97c-61cb7c5565ad None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:48.261206 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-3690fb36-eef8-4885-a97c-61cb7c5565ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:48.305110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00bbb2-9433-4fa7-ba7e-a90f8d67c1fe None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:48.345562 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 219/445] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:48.349301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241843e0-5f34-4ba7-b145-33eacb83f46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:48.349994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241843e0-5f34-4ba7-b145-33eacb83f46a None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:48.350327 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241843e0-5f34-4ba7-b145-33eacb83f46a None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:48.350651 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241843e0-5f34-4ba7-b145-33eacb83f46a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:48.369295 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690fb36-eef8-4885-a97c-61cb7c5565ad None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:48.438345 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:48.443031 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241843e0-5f34-4ba7-b145-33eacb83f46a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:48.445324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241843e0-5f34-4ba7-b145-33eacb83f46a None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:48.496799 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:48.515219 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690fb36-eef8-4885-a97c-61cb7c5565ad None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:48.562868 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfebb13-d2c3-4ff1-a986-38990bc6b147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:48.563522 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfebb13-d2c3-4ff1-a986-38990bc6b147 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:48.563801 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfebb13-d2c3-4ff1-a986-38990bc6b147 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:48.564070 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfebb13-d2c3-4ff1-a986-38990bc6b147 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:48.580305 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfebb13-d2c3-4ff1-a986-38990bc6b147 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:48.585384 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-fdfebb13-d2c3-4ff1-a986-38990bc6b147 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:49:48.585384 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:48.585384 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:48.585384 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:48.585384 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:48.585384 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:48.585384 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:48.585384 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:48.585384 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:48.585384 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:48.585384 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:48.585384 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:48.585384 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:48.585384 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:48.585384 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:48.585384 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:48.586291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:48.586291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:48.586291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:48.586291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:48.586291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:48.586291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:48.586291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:48.586291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:48.586291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:48.586291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:48.586291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:48.586291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:48.586291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:48.586291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:48.586291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:48.586291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:48.587148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:48.587148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:48.587148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:48.587148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:48.587148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:48.587148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:48.587148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:48.587148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:48.587148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:48.587148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:48.587148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:49:48.587148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:48.587748 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 227/446] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 19:49:48.589122 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 220/447] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:49:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:48.594675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241843e0-5f34-4ba7-b145-33eacb83f46a None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:48.597601 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfebb13-d2c3-4ff1-a986-38990bc6b147 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:48.720698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:48.725095 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241843e0-5f34-4ba7-b145-33eacb83f46a None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:48.726197 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfebb13-d2c3-4ff1-a986-38990bc6b147 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:48.770318 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d758331-c031-41cb-896b-c488239336c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:48.770974 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d758331-c031-41cb-896b-c488239336c5 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:48.771259 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d758331-c031-41cb-896b-c488239336c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:48.771538 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d758331-c031-41cb-896b-c488239336c5 None admin] PATH_INFO: `/v3/projects/c0ded24d8148467b9a56fa55332feb56` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:48.775660 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d758331-c031-41cb-896b-c488239336c5 None admin] RBAC: Authorizing `identity:get_project(project_id=c0ded24d8148467b9a56fa55332feb56)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:48.775660 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d758331-c031-41cb-896b-c488239336c5 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:48.775887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06006faa-eea6-40fc-9bc6-0ee781dcf64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:48.775887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06006faa-eea6-40fc-9bc6-0ee781dcf64f None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:48.775887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06006faa-eea6-40fc-9bc6-0ee781dcf64f None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:48.776107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06006faa-eea6-40fc-9bc6-0ee781dcf64f None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:48.776887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06006faa-eea6-40fc-9bc6-0ee781dcf64f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:48.777200 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06006faa-eea6-40fc-9bc6-0ee781dcf64f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:48.778604 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06006faa-eea6-40fc-9bc6-0ee781dcf64f None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:48.781427 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 228/448] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:48 2022] GET /identity/v3/projects/c0ded24d8148467b9a56fa55332feb56 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:48.783111 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d758331-c031-41cb-896b-c488239336c5 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:48.791288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 221/449] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:48.798740 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06006faa-eea6-40fc-9bc6-0ee781dcf64f None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:48.816242 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:48.888826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d758331-c031-41cb-896b-c488239336c5 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:48.906325 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06006faa-eea6-40fc-9bc6-0ee781dcf64f None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:48.924735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12fc02-b692-47fb-ade2-52dccf8ea575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:48.925373 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12fc02-b692-47fb-ade2-52dccf8ea575 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:48.925654 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12fc02-b692-47fb-ade2-52dccf8ea575 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:48.925970 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12fc02-b692-47fb-ade2-52dccf8ea575 None admin] PATH_INFO: `/v3/groups/f2e4c5e7f5df4798982e58481f941bfe` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:48.927063 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f12fc02-b692-47fb-ade2-52dccf8ea575 None admin] RBAC: Authorizing `identity:get_group(group_id=f2e4c5e7f5df4798982e58481f941bfe)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:48.928510 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f12fc02-b692-47fb-ade2-52dccf8ea575 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:48.935440 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 229/450] 10.222.0.39 () {56 vars in 1180 bytes} [Fri Aug 5 19:49:48 2022] GET /identity/v3/groups/f2e4c5e7f5df4798982e58481f941bfe => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:48.936212 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12fc02-b692-47fb-ade2-52dccf8ea575 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:48.946560 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e90537e-8949-4eb1-93de-49f925105434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:48.947171 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90537e-8949-4eb1-93de-49f925105434 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:48.947449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90537e-8949-4eb1-93de-49f925105434 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:48.947719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90537e-8949-4eb1-93de-49f925105434 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:48.971245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:49.019912 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e90537e-8949-4eb1-93de-49f925105434 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:49.021514 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e90537e-8949-4eb1-93de-49f925105434 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:49.038087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:49.039726 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12fc02-b692-47fb-ade2-52dccf8ea575 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:49.077060 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a6c90a-0533-443e-840e-ef49e57e0131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:49.078040 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6c90a-0533-443e-840e-ef49e57e0131 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:49.078315 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6c90a-0533-443e-840e-ef49e57e0131 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:49.078586 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6c90a-0533-443e-840e-ef49e57e0131 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:49.079380 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a6c90a-0533-443e-840e-ef49e57e0131 None admin] RBAC: Adding query filter params (group.id=f2e4c5e7f5df4798982e58481f941bfe, role.id=0eafd8a7ba2d47faaf5aecbb3360143b, scope.project.id=c0ded24d8148467b9a56fa55332feb56) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:49.079691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a6c90a-0533-443e-840e-ef49e57e0131 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:49.081188 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a6c90a-0533-443e-840e-ef49e57e0131 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:49.087513 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 230/451] 10.222.0.39 () {56 vars in 1376 bytes} [Fri Aug 5 19:49:48 2022] GET /identity/v3/role_assignments?group.id=f2e4c5e7f5df4798982e58481f941bfe&scope.project.id=c0ded24d8148467b9a56fa55332feb56&role.id=0eafd8a7ba2d47faaf5aecbb3360143b => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:49.164812 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 222/452] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 19:49:49.166081 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a6c90a-0533-443e-840e-ef49e57e0131 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:49.275645 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:49.281425 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a6c90a-0533-443e-840e-ef49e57e0131 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:49.315807 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f86f1-92fa-46d5-b182-b137267cf29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:49.316888 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f86f1-92fa-46d5-b182-b137267cf29a None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:49.316888 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f86f1-92fa-46d5-b182-b137267cf29a None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:49.317090 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f86f1-92fa-46d5-b182-b137267cf29a None admin] PATH_INFO: `/v3/projects/9bb3822e6d1c4f82815dbeda0f6eff52` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:49.318153 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291f86f1-92fa-46d5-b182-b137267cf29a None admin] RBAC: Authorizing `identity:get_project(project_id=9bb3822e6d1c4f82815dbeda0f6eff52)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:49.319576 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291f86f1-92fa-46d5-b182-b137267cf29a None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:49.321292 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 231/453] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:49 2022] GET /identity/v3/projects/9bb3822e6d1c4f82815dbeda0f6eff52 => generated 344 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:49.328486 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e90537e-8949-4eb1-93de-49f925105434 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:49.441761 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e90537e-8949-4eb1-93de-49f925105434 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:49.458271 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:49.479859 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0b223-b656-4ce8-b0ba-4b0d2a35c627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:49.480482 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0b223-b656-4ce8-b0ba-4b0d2a35c627 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:49.480796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0b223-b656-4ce8-b0ba-4b0d2a35c627 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:49.481058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0b223-b656-4ce8-b0ba-4b0d2a35c627 None admin] PATH_INFO: `/v3/groups/04484d0cd9c54c72bf4ae174e1cce60e` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:49.482175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a0b223-b656-4ce8-b0ba-4b0d2a35c627 None admin] RBAC: Authorizing `identity:get_group(group_id=04484d0cd9c54c72bf4ae174e1cce60e)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:49.483669 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a0b223-b656-4ce8-b0ba-4b0d2a35c627 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:49.485260 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 223/454] 10.222.0.39 () {56 vars in 1180 bytes} [Fri Aug 5 19:49:49 2022] GET /identity/v3/groups/04484d0cd9c54c72bf4ae174e1cce60e => generated 229 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:49.491234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f86f1-92fa-46d5-b182-b137267cf29a None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:49.591961 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f86f1-92fa-46d5-b182-b137267cf29a None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:49.624946 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6ee96-acd2-4745-a4fc-50e101466291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:49.625548 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6ee96-acd2-4745-a4fc-50e101466291 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:49.625828 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6ee96-acd2-4745-a4fc-50e101466291 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:49.626100 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6ee96-acd2-4745-a4fc-50e101466291 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:49.626887 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6ee96-acd2-4745-a4fc-50e101466291 None admin] RBAC: Adding query filter params (group.id=04484d0cd9c54c72bf4ae174e1cce60e, role.id=89919f784ad144529b07c8ab92ee7d48, scope.project.id=9bb3822e6d1c4f82815dbeda0f6eff52) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:49.627194 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6ee96-acd2-4745-a4fc-50e101466291 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:49.628649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6ee96-acd2-4745-a4fc-50e101466291 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:49.634461 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 232/455] 10.222.0.39 () {56 vars in 1376 bytes} [Fri Aug 5 19:49:49 2022] GET /identity/v3/role_assignments?group.id=04484d0cd9c54c72bf4ae174e1cce60e&scope.project.id=9bb3822e6d1c4f82815dbeda0f6eff52&role.id=89919f784ad144529b07c8ab92ee7d48 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:52.188528 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c266a0-9a0b-4cec-96e6-fb307a8c3c86 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:52.189055 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c266a0-9a0b-4cec-96e6-fb307a8c3c86 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:52.189129 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c266a0-9a0b-4cec-96e6-fb307a8c3c86 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:52.190718 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 224/456] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:52.196133 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25d546-bc6a-403b-973a-ed1c27963794 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:52.196759 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25d546-bc6a-403b-973a-ed1c27963794 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:52.196759 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25d546-bc6a-403b-973a-ed1c27963794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:52.222575 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-6a25d546-bc6a-403b-973a-ed1c27963794 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:52.223781 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-6a25d546-bc6a-403b-973a-ed1c27963794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:52.259347 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:52.319488 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 233/457] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:52.382018 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c266a0-9a0b-4cec-96e6-fb307a8c3c86 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:52.485962 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627851fa-9321-46c5-bd52-08559ac0ffae None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:52.486146 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627851fa-9321-46c5-bd52-08559ac0ffae None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:52.486435 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627851fa-9321-46c5-bd52-08559ac0ffae None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:52.488629 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 234/458] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:52.495346 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18206e98-2f11-40aa-bbb2-a66e2e28e885 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:52.495911 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18206e98-2f11-40aa-bbb2-a66e2e28e885 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:52.496383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18206e98-2f11-40aa-bbb2-a66e2e28e885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:52.526812 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c266a0-9a0b-4cec-96e6-fb307a8c3c86 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:52.533224 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-18206e98-2f11-40aa-bbb2-a66e2e28e885 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:52.534733 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-18206e98-2f11-40aa-bbb2-a66e2e28e885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:52.565227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50ad7a-2652-4a00-bfca-54b89c043b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:52.566507 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50ad7a-2652-4a00-bfca-54b89c043b51 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:52.566997 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50ad7a-2652-4a00-bfca-54b89c043b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:52.567662 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50ad7a-2652-4a00-bfca-54b89c043b51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:52.582561 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c50ad7a-2652-4a00-bfca-54b89c043b51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:52.584316 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c50ad7a-2652-4a00-bfca-54b89c043b51 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:52.614544 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-7c50ad7a-2652-4a00-bfca-54b89c043b51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:52.614544 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:52.614544 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:52.614544 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:52.614544 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:52.614544 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:52.614544 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:52.614544 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:52.614544 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:52.614544 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:52.614544 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:52.614544 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:52.614544 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:52.614544 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:52.614544 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:52.615483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:52.615483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:52.615483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:52.615483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:52.615483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:52.615483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:52.615483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:52.615483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:52.615483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:52.615483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:52.615483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:52.615483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:52.615483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:52.615483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:52.615483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:52.615483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:52.615483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:52.616308 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:52.616308 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:52.616308 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:52.616308 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:52.618828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 225/459] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:52.620162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f8258-a21b-463e-adfb-9455f1a69f4a None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:52.620569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f8258-a21b-463e-adfb-9455f1a69f4a None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:52.620994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f8258-a21b-463e-adfb-9455f1a69f4a None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:52.622629 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 226/460] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:52.625219 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f8258-a21b-463e-adfb-9455f1a69f4a None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:52.634254 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 235/461] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:52.635704 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc5b59-ea02-408a-847d-8b5431aa1389 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:52.636068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc5b59-ea02-408a-847d-8b5431aa1389 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:52.636422 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc5b59-ea02-408a-847d-8b5431aa1389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:52.656509 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-a0dc5b59-ea02-408a-847d-8b5431aa1389 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:52.657845 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-a0dc5b59-ea02-408a-847d-8b5431aa1389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:52.673128 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:52.769035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 236/462] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:52.770140 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc5b59-ea02-408a-847d-8b5431aa1389 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:52.790851 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f8258-a21b-463e-adfb-9455f1a69f4a None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:52.795424 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:52.855608 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2367d6a4-48c4-46b6-a1ea-6960f204857c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:52.856350 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367d6a4-48c4-46b6-a1ea-6960f204857c None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:52.856777 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367d6a4-48c4-46b6-a1ea-6960f204857c None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:52.857947 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367d6a4-48c4-46b6-a1ea-6960f204857c None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:52.859288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2367d6a4-48c4-46b6-a1ea-6960f204857c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:52.859890 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2367d6a4-48c4-46b6-a1ea-6960f204857c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:52.861689 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2367d6a4-48c4-46b6-a1ea-6960f204857c None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:52.880845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 227/463] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:52.882144 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2367d6a4-48c4-46b6-a1ea-6960f204857c None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:52.913253 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc5b59-ea02-408a-847d-8b5431aa1389 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:52.950877 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb0b18-22fa-4178-b3b9-135a07b282e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:52.952524 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb0b18-22fa-4178-b3b9-135a07b282e8 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:52.953033 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb0b18-22fa-4178-b3b9-135a07b282e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:52.953500 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb0b18-22fa-4178-b3b9-135a07b282e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:52.965439 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb0b18-22fa-4178-b3b9-135a07b282e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:52.967738 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb0b18-22fa-4178-b3b9-135a07b282e8 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:52.986313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-45fb0b18-22fa-4178-b3b9-135a07b282e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:52.986313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:52.986313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:52.986313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:52.986313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:52.986313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:52.986313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:52.986313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:52.986313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:52.986313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:52.986313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:52.986313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:52.986313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:52.986313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:52.986313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:52.987262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:52.987262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:52.987262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:52.987262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:52.987262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:52.987262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:52.987262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:52.987262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:52.987262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:52.987262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:52.987262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:52.987262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:52.987262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:52.987262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:52.987262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:52.987262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:52.987262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:52.988130 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:52.988130 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:52.988130 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:52.988130 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:49:52.995234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 237/464] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:52.996435 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb0b18-22fa-4178-b3b9-135a07b282e8 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:53.030879 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:53.044494 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2367d6a4-48c4-46b6-a1ea-6960f204857c None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:53.095439 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2983ea-30eb-467f-87cd-be8105e1cf0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:53.095714 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:53.096084 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2983ea-30eb-467f-87cd-be8105e1cf0c None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:53.096411 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2983ea-30eb-467f-87cd-be8105e1cf0c None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:53.096702 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2983ea-30eb-467f-87cd-be8105e1cf0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:53.108430 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2983ea-30eb-467f-87cd-be8105e1cf0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:53.109936 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2983ea-30eb-467f-87cd-be8105e1cf0c None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:53.128085 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-de2983ea-30eb-467f-87cd-be8105e1cf0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:53.128085 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:53.128085 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:53.128085 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:53.128085 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:53.128085 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:53.128085 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:53.128085 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:53.128085 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:53.128085 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:53.128085 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:53.128085 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:53.128085 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:53.128085 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:53.128085 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:53.128994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:53.128994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:53.128994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:53.128994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:53.128994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:53.128994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:53.128994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:53.128994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:53.128994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:53.128994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:53.128994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:53.128994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:53.128994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:53.128994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:53.128994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:53.128994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:53.128994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:53.129823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:53.129823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:53.129823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:53.129823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:53.129823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 228/465] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:53.130075 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2983ea-30eb-467f-87cd-be8105e1cf0c None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:53.142491 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb0b18-22fa-4178-b3b9-135a07b282e8 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:53.193698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e967fa-9bba-4ea2-abb4-2e58cf7f6547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:53.194589 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e967fa-9bba-4ea2-abb4-2e58cf7f6547 None admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:53.194868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e967fa-9bba-4ea2-abb4-2e58cf7f6547 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:53.195139 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e967fa-9bba-4ea2-abb4-2e58cf7f6547 None admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:53.195820 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e967fa-9bba-4ea2-abb4-2e58cf7f6547 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:53.198147 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e967fa-9bba-4ea2-abb4-2e58cf7f6547 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:53.223456 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 238/466] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Aug 5 19:49:52 2022] POST /identity/v3/users => generated 246 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:53.225024 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be17377-4061-41be-93e0-db1ee4eec1b2 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:53.225312 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be17377-4061-41be-93e0-db1ee4eec1b2 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:53.225589 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be17377-4061-41be-93e0-db1ee4eec1b2 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:53.231604 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 239/467] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:53.231604 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be17377-4061-41be-93e0-db1ee4eec1b2 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:53.259231 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2983ea-30eb-467f-87cd-be8105e1cf0c None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:53.312523 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40674371-c713-49fe-837c-761b79d1d65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:53.314086 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40674371-c713-49fe-837c-761b79d1d65a None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:53.314899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40674371-c713-49fe-837c-761b79d1d65a None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:53.315812 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40674371-c713-49fe-837c-761b79d1d65a None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:53.317598 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40674371-c713-49fe-837c-761b79d1d65a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:53.318462 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40674371-c713-49fe-837c-761b79d1d65a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:53.321815 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40674371-c713-49fe-837c-761b79d1d65a None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:53.341383 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 229/468] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:53.344510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4cb61c-9efd-495a-9ece-adc3303c2c8c None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:53.345122 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4cb61c-9efd-495a-9ece-adc3303c2c8c None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:53.345596 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4cb61c-9efd-495a-9ece-adc3303c2c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:53.354990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:53.371305 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be17377-4061-41be-93e0-db1ee4eec1b2 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:53.376561 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-8f4cb61c-9efd-495a-9ece-adc3303c2c8c None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:53.378278 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-8f4cb61c-9efd-495a-9ece-adc3303c2c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:53.413761 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:53.426483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab3777-fa71-4aef-afb2-1da9b85f34f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:53.427262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab3777-fa71-4aef-afb2-1da9b85f34f0 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:53.427545 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab3777-fa71-4aef-afb2-1da9b85f34f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:53.427820 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab3777-fa71-4aef-afb2-1da9b85f34f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:53.428603 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ab3777-fa71-4aef-afb2-1da9b85f34f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:53.428921 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ab3777-fa71-4aef-afb2-1da9b85f34f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:53.430354 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ab3777-fa71-4aef-afb2-1da9b85f34f0 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:53.454350 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 240/469] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:53.456296 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab3777-fa71-4aef-afb2-1da9b85f34f0 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:53.493820 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 230/470] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:53.494686 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4cb61c-9efd-495a-9ece-adc3303c2c8c None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:53.607054 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab3777-fa71-4aef-afb2-1da9b85f34f0 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:53.632618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4cb61c-9efd-495a-9ece-adc3303c2c8c None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:53.653314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eaf329-e765-44e6-907d-ddd274688c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:53.653962 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eaf329-e765-44e6-907d-ddd274688c93 None admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:53.654629 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eaf329-e765-44e6-907d-ddd274688c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:53.655127 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eaf329-e765-44e6-907d-ddd274688c93 None admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:53.655852 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eaf329-e765-44e6-907d-ddd274688c93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:53.657358 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eaf329-e765-44e6-907d-ddd274688c93 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:53.674473 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb770722-7f4f-466f-aed3-a4dc17b9973d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:53.675544 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb770722-7f4f-466f-aed3-a4dc17b9973d None admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:53.676013 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb770722-7f4f-466f-aed3-a4dc17b9973d None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:53.676519 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb770722-7f4f-466f-aed3-a4dc17b9973d None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:53.677476 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb770722-7f4f-466f-aed3-a4dc17b9973d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:53.679203 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb770722-7f4f-466f-aed3-a4dc17b9973d None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:53.681637 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 241/471] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Aug 5 19:49:53 2022] POST /identity/v3/users => generated 248 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:53.682620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eaf329-e765-44e6-907d-ddd274688c93 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:53.689785 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:53.708695 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 231/472] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Aug 5 19:49:53 2022] POST /identity/v3/users => generated 248 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:53.766222 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:53.846295 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eaf329-e765-44e6-907d-ddd274688c93 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:53.885190 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423725a0-9a60-4e61-9f72-faad370c9e8f None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:53.885559 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423725a0-9a60-4e61-9f72-faad370c9e8f None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:53.885815 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423725a0-9a60-4e61-9f72-faad370c9e8f None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:53.887560 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 232/473] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:53.894484 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c372a9-cf33-4a94-b7bb-74983edab1bd None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:53.894804 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c372a9-cf33-4a94-b7bb-74983edab1bd None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:53.895037 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c372a9-cf33-4a94-b7bb-74983edab1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:53.900337 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57cc60-3442-4fcf-a217-93ba7456414c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:53.901428 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57cc60-3442-4fcf-a217-93ba7456414c None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:53.901919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57cc60-3442-4fcf-a217-93ba7456414c None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:53.902398 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57cc60-3442-4fcf-a217-93ba7456414c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:53.917562 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57cc60-3442-4fcf-a217-93ba7456414c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:53.919384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57cc60-3442-4fcf-a217-93ba7456414c None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:53.936783 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-fa57cc60-3442-4fcf-a217-93ba7456414c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:53.936783 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:53.936783 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:53.936783 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:53.936783 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:53.936783 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:53.936783 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:53.936783 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:53.936783 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:53.936783 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:53.936783 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:53.936783 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:53.936783 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:53.936783 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:53.936783 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:53.937744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:53.937744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:53.937744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:53.937744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:53.937744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:53.937744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:53.937744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:53.937744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:53.937744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:53.937744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:53.937744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:53.937744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:53.937744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:53.937744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:53.937744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:53.937744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:53.937744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:53.938603 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:53.938603 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:53.938603 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:53.938603 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:49:53.940892 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 242/474] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:53.943319 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-e0c372a9-cf33-4a94-b7bb-74983edab1bd None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:53.944541 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-e0c372a9-cf33-4a94-b7bb-74983edab1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:53.950900 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57cc60-3442-4fcf-a217-93ba7456414c None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:54.043139 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 233/475] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:54.066432 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c372a9-cf33-4a94-b7bb-74983edab1bd None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:54.076914 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:54.084855 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57cc60-3442-4fcf-a217-93ba7456414c None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:54.132595 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a20e49b-5c47-41aa-b078-bf7f1d09d321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:54.133778 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20e49b-5c47-41aa-b078-bf7f1d09d321 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:54.134667 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20e49b-5c47-41aa-b078-bf7f1d09d321 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:54.135161 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20e49b-5c47-41aa-b078-bf7f1d09d321 None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:54.136125 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a20e49b-5c47-41aa-b078-bf7f1d09d321 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:54.136638 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a20e49b-5c47-41aa-b078-bf7f1d09d321 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:54.139388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a20e49b-5c47-41aa-b078-bf7f1d09d321 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:54.158102 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 243/476] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:54.164506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a20e49b-5c47-41aa-b078-bf7f1d09d321 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:54.196902 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c372a9-cf33-4a94-b7bb-74983edab1bd None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:54.220911 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:54.237714 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd96ca6-af62-4463-9878-8eb10378b48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:54.238348 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd96ca6-af62-4463-9878-8eb10378b48b None admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:54.239099 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd96ca6-af62-4463-9878-8eb10378b48b None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:54.239099 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd96ca6-af62-4463-9878-8eb10378b48b None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:54.239772 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd96ca6-af62-4463-9878-8eb10378b48b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:54.241287 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd96ca6-af62-4463-9878-8eb10378b48b None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:54.251605 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.sql.core [None req-7bd96ca6-af62-4463-9878-8eb10378b48b None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 05 19:49:54.251605 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 05 19:49:54.251605 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [parameters: {'id': '2f14df977398456da591f0233e81f189', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 05 19:49:54.251605 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80477) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 05 19:49:54.276359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-7bd96ca6-af62-4463-9878-8eb10378b48b None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 05 19:49:54.276359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:54.276359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 19:49:54.276359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 19:49:54.276359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 19:49:54.276359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 19:49:54.276359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 19:49:54.276359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 19:49:54.276359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 19:49:54.276359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application conn.query(q) Aug 05 19:49:54.276359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 19:49:54.276359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 19:49:54.276359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 19:49:54.276359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application result.read() Aug 05 19:49:54.276359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 19:49:54.277899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 19:49:54.277899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 19:49:54.277899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 19:49:54.277899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 19:49:54.277899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 19:49:54.277899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 19:49:54.277899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 19:49:54.277899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 05 19:49:54.277899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:54.277899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 05 19:49:54.277899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:54.277899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:54.277899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 05 19:49:54.277899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 05 19:49:54.277899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 05 19:49:54.277899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return ref.to_dict() Aug 05 19:49:54.277899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 19:49:54.277899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application next(self.gen) Aug 05 19:49:54.279084 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 05 19:49:54.279084 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application yield resource Aug 05 19:49:54.279084 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 19:49:54.279084 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application next(self.gen) Aug 05 19:49:54.279084 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 05 19:49:54.279084 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self.session.rollback() Aug 05 19:49:54.279084 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 05 19:49:54.279084 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self.force_reraise() Aug 05 19:49:54.279084 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 05 19:49:54.279084 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise self.value Aug 05 19:49:54.279084 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 05 19:49:54.279084 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 05 19:49:54.279084 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 05 19:49:54.279084 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application session.commit() Aug 05 19:49:54.279084 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 05 19:49:54.279084 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 05 19:49:54.279084 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 05 19:49:54.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self._prepare_impl() Aug 05 19:49:54.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 05 19:49:54.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self.session.flush() Aug 05 19:49:54.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 05 19:49:54.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self._flush(objects) Aug 05 19:49:54.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 05 19:49:54.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 05 19:49:54.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 05 19:49:54.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application compat.raise_( Aug 05 19:49:54.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 19:49:54.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception Aug 05 19:49:54.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 05 19:49:54.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application flush_context.execute() Aug 05 19:49:54.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 05 19:49:54.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rec.execute(self) Aug 05 19:49:54.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 05 19:49:54.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 05 19:49:54.286138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 05 19:49:54.286138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 05 19:49:54.286138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 05 19:49:54.286138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 05 19:49:54.286138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 05 19:49:54.286138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 05 19:49:54.286138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 05 19:49:54.286138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 05 19:49:54.286138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 05 19:49:54.286138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 05 19:49:54.286138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 05 19:49:54.286138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 05 19:49:54.286138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 05 19:49:54.286138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 05 19:49:54.286138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 19:49:54.286138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception Aug 05 19:49:54.288252 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 19:49:54.288252 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 19:49:54.288252 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 19:49:54.288252 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 19:49:54.288252 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 19:49:54.288252 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 19:49:54.288252 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 19:49:54.288252 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application conn.query(q) Aug 05 19:49:54.288252 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 19:49:54.288252 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 19:49:54.288252 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 19:49:54.288252 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application result.read() Aug 05 19:49:54.288252 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 19:49:54.288252 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 19:49:54.288252 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 19:49:54.288252 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 19:49:54.290215 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 19:49:54.290215 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 19:49:54.290215 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 19:49:54.290215 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 19:49:54.290215 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 05 19:49:54.290215 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 05 19:49:54.290215 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [parameters: {'id': '2f14df977398456da591f0233e81f189', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 05 19:49:54.290215 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 05 19:49:54.290215 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:54.290215 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:49:54.290215 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:54.290215 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:54.290215 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:54.290215 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:54.290215 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:54.290215 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:54.292359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:54.292359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:54.292359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:54.292359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:54.292359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:54.292359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:54.292359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 05 19:49:54.292359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 05 19:49:54.292359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:54.292359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:54.292359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 05 19:49:54.292359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 05 19:49:54.292359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 05 19:49:54.292359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 05 19:49:54.292359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 05 19:49:54.292359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:54.294445 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 234/477] 10.222.0.39 () {60 vars in 1131 bytes} [Fri Aug 5 19:49:54 2022] POST /identity/v3/roles => generated 144 bytes in 214 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:54.294588 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a20e49b-5c47-41aa-b078-bf7f1d09d321 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:54.295588 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd96ca6-af62-4463-9878-8eb10378b48b None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:54.352038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae188e72-8e41-4042-86f0-e89c5efe7284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:54.352782 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae188e72-8e41-4042-86f0-e89c5efe7284 None admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:54.353066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae188e72-8e41-4042-86f0-e89c5efe7284 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:54.353342 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae188e72-8e41-4042-86f0-e89c5efe7284 None admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:54.354037 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae188e72-8e41-4042-86f0-e89c5efe7284 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:54.355557 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae188e72-8e41-4042-86f0-e89c5efe7284 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:54.388723 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 244/478] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Aug 5 19:49:54 2022] POST /identity/v3/users => generated 249 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:54.468928 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd96ca6-af62-4463-9878-8eb10378b48b None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:54.527543 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1a0bf-8a4d-432e-84f1-b01d5ea493a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:54.528204 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1a0bf-8a4d-432e-84f1-b01d5ea493a1 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:54.528486 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1a0bf-8a4d-432e-84f1-b01d5ea493a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:54.528781 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1a0bf-8a4d-432e-84f1-b01d5ea493a1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:54.554098 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1a0bf-8a4d-432e-84f1-b01d5ea493a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:54.558940 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-3bc1a0bf-8a4d-432e-84f1-b01d5ea493a1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:54.558940 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:54.558940 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:54.558940 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:54.558940 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:54.558940 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:54.558940 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:54.558940 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:54.558940 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:54.558940 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:54.558940 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:54.558940 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:54.558940 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:54.558940 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:54.558940 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:54.558940 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:54.559847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:54.559847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:54.559847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:54.559847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:54.559847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:54.559847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:54.559847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:54.559847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:54.559847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:54.559847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:54.559847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:54.559847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:54.559847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:54.559847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:54.559847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:54.559847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:54.561313 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:54.561313 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:54.561313 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:54.561313 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:54.561313 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:54.561313 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:54.561313 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:54.561313 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:54.561313 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:54.561313 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:54.561313 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:54.561313 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:54.561917 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 235/479] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Aug 5 19:49:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:54.567001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae188e72-8e41-4042-86f0-e89c5efe7284 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:54.587764 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:54.689750 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae188e72-8e41-4042-86f0-e89c5efe7284 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:54.729327 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b0198-5ffa-4d5b-b9d9-e247916fdb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:54.730243 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b0198-5ffa-4d5b-b9d9-e247916fdb13 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:54.730697 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b0198-5ffa-4d5b-b9d9-e247916fdb13 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:54.731140 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b0198-5ffa-4d5b-b9d9-e247916fdb13 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:54.732061 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08b0198-5ffa-4d5b-b9d9-e247916fdb13 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:54.732649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08b0198-5ffa-4d5b-b9d9-e247916fdb13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:54.734502 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08b0198-5ffa-4d5b-b9d9-e247916fdb13 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:54.743803 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 245/480] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:56.202441 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13179bfa-7899-4342-9497-81bc06cbcd5f None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:56.202982 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13179bfa-7899-4342-9497-81bc06cbcd5f None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:56.202982 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13179bfa-7899-4342-9497-81bc06cbcd5f None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:56.204760 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 236/481] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:56.210121 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f7f99-3bf6-4a64-a839-25b6c2e20286 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:56.210530 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f7f99-3bf6-4a64-a839-25b6c2e20286 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:56.210866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f7f99-3bf6-4a64-a839-25b6c2e20286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:56.231571 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-918f7f99-3bf6-4a64-a839-25b6c2e20286 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:56.238222 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-918f7f99-3bf6-4a64-a839-25b6c2e20286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:56.324085 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 246/482] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:56.348365 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13179bfa-7899-4342-9497-81bc06cbcd5f None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:56.423400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:56.453341 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe005740-d0d1-45e8-9cfc-d97da1be3ce7 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:56.453655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe005740-d0d1-45e8-9cfc-d97da1be3ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:56.453895 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe005740-d0d1-45e8-9cfc-d97da1be3ce7 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:56.455531 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 247/483] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:56.460924 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858f860-a7aa-43b2-852a-df6cbef0da84 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:56.461452 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858f860-a7aa-43b2-852a-df6cbef0da84 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:56.461873 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858f860-a7aa-43b2-852a-df6cbef0da84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:56.471339 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:56.484160 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-0858f860-a7aa-43b2-852a-df6cbef0da84 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:56.485752 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-0858f860-a7aa-43b2-852a-df6cbef0da84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:56.534167 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13179bfa-7899-4342-9497-81bc06cbcd5f None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:56.620422 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 248/484] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:49:56.638806 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d100ebc-f3bd-420a-8287-556e640bf37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:56.639474 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d100ebc-f3bd-420a-8287-556e640bf37e None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:56.639760 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d100ebc-f3bd-420a-8287-556e640bf37e None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:56.640057 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d100ebc-f3bd-420a-8287-556e640bf37e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:56.646136 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0858f860-a7aa-43b2-852a-df6cbef0da84 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:56.669131 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d100ebc-f3bd-420a-8287-556e640bf37e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:56.679682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-9d100ebc-f3bd-420a-8287-556e640bf37e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:56.679682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:56.679682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:56.679682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:56.679682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:56.679682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:56.679682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:56.679682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:56.679682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:56.679682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:56.679682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:56.679682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:56.679682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:56.679682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:56.679682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:56.679682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:56.680645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:56.680645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:56.680645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:56.680645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:56.680645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:56.680645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:56.680645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:56.680645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:56.680645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:56.680645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:56.680645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:56.680645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:56.680645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:56.680645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:56.680645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:56.680645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:56.681442 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:56.681442 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:56.681442 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:56.681442 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:56.681442 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:56.681442 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:56.681442 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:56.681442 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:56.681442 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:56.681442 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:56.681442 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:56.681442 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:56.688233 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 237/485] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Aug 5 19:49:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:56.703087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d100ebc-f3bd-420a-8287-556e640bf37e None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:56.854501 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d100ebc-f3bd-420a-8287-556e640bf37e None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:56.858907 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:56.886269 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0858f860-a7aa-43b2-852a-df6cbef0da84 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:56.902049 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80fccdf-382c-4b2c-bfbd-c929d46f10e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:56.902666 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fccdf-382c-4b2c-bfbd-c929d46f10e3 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:56.902951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fccdf-382c-4b2c-bfbd-c929d46f10e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:56.903232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fccdf-382c-4b2c-bfbd-c929d46f10e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:56.904048 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80fccdf-382c-4b2c-bfbd-c929d46f10e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:56.904975 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80fccdf-382c-4b2c-bfbd-c929d46f10e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:56.906432 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80fccdf-382c-4b2c-bfbd-c929d46f10e3 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:56.925569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 238/486] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Aug 5 19:49:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:56.926824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b44b8c1-4653-472c-85e5-4d53d8dd0028 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:56.927124 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b44b8c1-4653-472c-85e5-4d53d8dd0028 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:56.927400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b44b8c1-4653-472c-85e5-4d53d8dd0028 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:56.929043 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 239/487] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:56.929446 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b44b8c1-4653-472c-85e5-4d53d8dd0028 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:56.961677 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be1c3d-f74d-46d2-8934-e6810d66a788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:56.963348 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be1c3d-f74d-46d2-8934-e6810d66a788 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:56.964287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be1c3d-f74d-46d2-8934-e6810d66a788 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:56.965258 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be1c3d-f74d-46d2-8934-e6810d66a788 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:56.976923 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:56.989512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be1c3d-f74d-46d2-8934-e6810d66a788 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:57.007290 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-05be1c3d-f74d-46d2-8934-e6810d66a788 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:57.007290 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:57.007290 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:57.007290 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:57.007290 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:57.007290 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:57.007290 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:57.007290 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:57.007290 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:57.007290 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:57.007290 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:57.007290 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:57.007290 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:57.007290 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:57.007290 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:57.007290 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:57.008430 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:57.008430 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:57.008430 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:57.008430 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:57.008430 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:57.008430 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:57.008430 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:57.008430 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:57.008430 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:57.008430 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:57.008430 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:57.008430 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:57.008430 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:57.008430 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:57.008430 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:57.008430 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:57.009293 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:57.009293 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:57.009293 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:57.009293 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:57.009293 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:57.009293 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:57.009293 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:57.009293 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:57.009293 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:57.009293 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:57.009293 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:57.009293 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:49:57.016540 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 249/488] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Aug 5 19:49:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:57.016540 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c7e10-3623-4313-bda0-0ec4eb200d26 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:57.016540 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c7e10-3623-4313-bda0-0ec4eb200d26 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:57.016540 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c7e10-3623-4313-bda0-0ec4eb200d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:57.041022 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-954c7e10-3623-4313-bda0-0ec4eb200d26 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:57.042363 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-954c7e10-3623-4313-bda0-0ec4eb200d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:57.076515 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b44b8c1-4653-472c-85e5-4d53d8dd0028 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:57.130349 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36f50f-841f-4f4a-910b-ffcd8621a0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:57.131410 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36f50f-841f-4f4a-910b-ffcd8621a0af None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:57.131890 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36f50f-841f-4f4a-910b-ffcd8621a0af None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:57.132415 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36f50f-841f-4f4a-910b-ffcd8621a0af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:57.154308 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba36f50f-841f-4f4a-910b-ffcd8621a0af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:57.157056 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba36f50f-841f-4f4a-910b-ffcd8621a0af None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:57.163177 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 250/489] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:57.164830 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c7e10-3623-4313-bda0-0ec4eb200d26 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:57.176232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-ba36f50f-841f-4f4a-910b-ffcd8621a0af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:57.176232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:57.176232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:57.176232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:57.176232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:57.176232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:57.176232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:57.176232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:57.176232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:57.176232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:57.176232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:57.176232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:57.176232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:57.176232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:57.176232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:57.177758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:57.177758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:57.177758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:57.177758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:57.177758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:57.177758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:57.177758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:57.177758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:57.177758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:57.177758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:57.177758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:57.177758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:57.177758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:57.177758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:57.177758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:57.177758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:57.177758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:57.178617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:57.178617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:57.178617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:57.178617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:57.181292 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 240/490] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:57.182817 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164c8e8-8490-4efe-b559-cd5a25b69861 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:57.183283 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164c8e8-8490-4efe-b559-cd5a25b69861 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:57.183894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164c8e8-8490-4efe-b559-cd5a25b69861 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:57.186272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 241/491] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:57.188094 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164c8e8-8490-4efe-b559-cd5a25b69861 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:57.217873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:57.309886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c7e10-3623-4313-bda0-0ec4eb200d26 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:57.328002 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164c8e8-8490-4efe-b559-cd5a25b69861 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:57.340999 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:57.355392 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad4d05a-fb0e-4455-8b7f-73fe0abc84b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:57.356020 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4d05a-fb0e-4455-8b7f-73fe0abc84b1 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:57.356304 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4d05a-fb0e-4455-8b7f-73fe0abc84b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:57.356612 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4d05a-fb0e-4455-8b7f-73fe0abc84b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:57.357911 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad4d05a-fb0e-4455-8b7f-73fe0abc84b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:57.358301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad4d05a-fb0e-4455-8b7f-73fe0abc84b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:57.359727 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad4d05a-fb0e-4455-8b7f-73fe0abc84b1 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:57.373972 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 251/492] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Aug 5 19:49:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:57.373972 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad4d05a-fb0e-4455-8b7f-73fe0abc84b1 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:57.376747 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db3b13-f362-4bc4-a6d4-a49eeda3cb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:57.377407 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db3b13-f362-4bc4-a6d4-a49eeda3cb3d None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:57.377691 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db3b13-f362-4bc4-a6d4-a49eeda3cb3d None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:57.377977 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db3b13-f362-4bc4-a6d4-a49eeda3cb3d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:57.392815 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36db3b13-f362-4bc4-a6d4-a49eeda3cb3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:57.398330 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-36db3b13-f362-4bc4-a6d4-a49eeda3cb3d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:57.398330 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:57.398330 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:57.398330 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:57.398330 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:57.398330 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:57.398330 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:57.398330 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:57.398330 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:57.398330 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:57.398330 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:57.398330 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:57.398330 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:57.398330 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:57.398330 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:57.398330 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:57.399272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:57.399272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:57.399272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:57.399272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:57.399272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:57.399272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:57.399272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:57.399272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:57.399272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:57.399272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:57.399272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:57.399272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:57.399272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:57.399272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:57.399272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:57.399272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:57.401031 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:57.401031 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:57.401031 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:57.401031 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:57.401031 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:57.401031 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:57.401031 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:57.401031 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:57.401031 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:57.401031 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:57.401031 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:57.401031 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:57.401031 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 242/493] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Aug 5 19:49:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:57.402067 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6a6ab-d365-4beb-9fdc-7b8cfe89b798 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:57.402067 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6a6ab-d365-4beb-9fdc-7b8cfe89b798 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:57.402067 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6a6ab-d365-4beb-9fdc-7b8cfe89b798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:57.421838 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-d8f6a6ab-d365-4beb-9fdc-7b8cfe89b798 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:57.423049 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-d8f6a6ab-d365-4beb-9fdc-7b8cfe89b798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:57.504470 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 243/494] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:57.506042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6a6ab-d365-4beb-9fdc-7b8cfe89b798 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:57.522361 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:57.531046 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad4d05a-fb0e-4455-8b7f-73fe0abc84b1 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:57.578216 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae9a1c-3294-4b93-b627-7b6e3fc67bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:57.578987 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae9a1c-3294-4b93-b627-7b6e3fc67bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:57.579276 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae9a1c-3294-4b93-b627-7b6e3fc67bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:57.579897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae9a1c-3294-4b93-b627-7b6e3fc67bf8 None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:57.580255 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ae9a1c-3294-4b93-b627-7b6e3fc67bf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:57.580564 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ae9a1c-3294-4b93-b627-7b6e3fc67bf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:57.581969 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ae9a1c-3294-4b93-b627-7b6e3fc67bf8 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:57.595854 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 252/495] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:57.595854 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae9a1c-3294-4b93-b627-7b6e3fc67bf8 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:57.645997 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6a6ab-d365-4beb-9fdc-7b8cfe89b798 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:57.653157 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:57.689247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c8a59-fad9-4b05-b15d-df89dfe4137c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:57.689881 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c8a59-fad9-4b05-b15d-df89dfe4137c None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:57.690155 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c8a59-fad9-4b05-b15d-df89dfe4137c None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:57.690488 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c8a59-fad9-4b05-b15d-df89dfe4137c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:57.701276 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898c8a59-fad9-4b05-b15d-df89dfe4137c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:57.702726 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898c8a59-fad9-4b05-b15d-df89dfe4137c None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:57.712020 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae9a1c-3294-4b93-b627-7b6e3fc67bf8 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:57.719183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-898c8a59-fad9-4b05-b15d-df89dfe4137c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:57.719183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:57.719183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:57.719183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:57.719183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:57.719183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:57.719183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:57.719183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:57.719183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:57.719183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:57.719183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:57.719183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:57.719183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:57.719183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:57.719183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:57.720182 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:57.720182 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:57.720182 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:57.720182 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:57.720182 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:57.720182 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:57.720182 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:57.720182 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:57.720182 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:57.720182 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:57.720182 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:57.720182 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:57.720182 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:57.720182 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:57.720182 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:57.720182 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:57.720182 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:57.725915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:57.725915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:57.725915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:57.725915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:57.725915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 244/496] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:57.725915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c8a59-fad9-4b05-b15d-df89dfe4137c None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:57.751167 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b71e45-759b-4d93-85b6-2d6facecc6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:57.751783 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b71e45-759b-4d93-85b6-2d6facecc6db None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:57.752067 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b71e45-759b-4d93-85b6-2d6facecc6db None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:57.752343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b71e45-759b-4d93-85b6-2d6facecc6db None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:57.753086 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b71e45-759b-4d93-85b6-2d6facecc6db None admin] RBAC: Adding query filter params (name=service) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:57.753396 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b71e45-759b-4d93-85b6-2d6facecc6db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:57.754839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b71e45-759b-4d93-85b6-2d6facecc6db None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:57.764394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 253/497] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Aug 5 19:49:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:57.766128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b71e45-759b-4d93-85b6-2d6facecc6db None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:57.784249 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:57.836915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c8a59-fad9-4b05-b15d-df89dfe4137c None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:57.873162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9bcba5-425d-4e10-81b8-41921d8bb93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:57.873751 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bcba5-425d-4e10-81b8-41921d8bb93a None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:57.874031 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bcba5-425d-4e10-81b8-41921d8bb93a None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:57.874302 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bcba5-425d-4e10-81b8-41921d8bb93a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:57.877892 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b71e45-759b-4d93-85b6-2d6facecc6db None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:57.886537 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9bcba5-425d-4e10-81b8-41921d8bb93a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:57.887718 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:57.891988 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-0b9bcba5-425d-4e10-81b8-41921d8bb93a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:57.891988 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:57.891988 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:57.891988 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:57.891988 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:57.891988 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:57.891988 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:57.891988 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:57.891988 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:57.891988 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:57.891988 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:57.891988 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:57.891988 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:57.891988 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:57.891988 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:57.891988 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:57.893939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:57.893939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:57.893939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:57.893939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:57.893939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:57.893939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:57.893939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:57.893939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:57.893939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:57.893939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:57.893939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:57.893939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:57.893939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:57.893939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:57.893939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:57.893939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:57.895768 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:57.895768 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:57.895768 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:57.895768 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:57.895768 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:57.895768 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:57.895768 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:57.895768 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:57.895768 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:57.895768 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:57.895768 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:57.895768 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:57.895768 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 245/498] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Aug 5 19:49:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:57.895768 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffb02d-0e71-4a32-aae7-bbfbef7a30da None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:57.898035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffb02d-0e71-4a32-aae7-bbfbef7a30da None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:57.898035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffb02d-0e71-4a32-aae7-bbfbef7a30da None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:57.898035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 246/499] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:49:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:57.898035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbffb02d-0e71-4a32-aae7-bbfbef7a30da None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:57.924341 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e41a2d-cad8-491f-8200-adb7464cb47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:57.925006 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e41a2d-cad8-491f-8200-adb7464cb47b None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:57.925285 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e41a2d-cad8-491f-8200-adb7464cb47b None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:57.925907 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e41a2d-cad8-491f-8200-adb7464cb47b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:57.933279 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e41a2d-cad8-491f-8200-adb7464cb47b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:57.934718 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e41a2d-cad8-491f-8200-adb7464cb47b None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:57.947568 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-74e41a2d-cad8-491f-8200-adb7464cb47b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:49:57.947568 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:57.947568 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:57.947568 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:57.947568 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:57.947568 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:57.947568 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:57.947568 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:57.947568 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:57.947568 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:57.947568 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:57.947568 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:57.947568 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:49:57.947568 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:49:57.947568 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:49:57.947568 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:49:57.949469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:57.949469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:57.949469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:49:57.949469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:57.949469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:49:57.949469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:57.949469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:57.949469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:57.949469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:57.949469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:57.949469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:57.949469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:57.949469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:57.949469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:57.949469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:57.949469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:57.949469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:57.949469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:57.951284 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:57.951284 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:57.951284 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:49:57.951284 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:49:57.951284 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:49:57.951284 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:49:57.951284 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:49:57.951284 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:49:57.951284 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:49:57.951284 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:49:57.951284 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 254/500] 10.222.0.39 () {56 vars in 1093 bytes} [Fri Aug 5 19:49:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:57.951284 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e41a2d-cad8-491f-8200-adb7464cb47b None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:58.015385 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbffb02d-0e71-4a32-aae7-bbfbef7a30da None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:58.036832 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:58.051785 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a47d54-0db2-47b0-94af-83ad2fde666c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:58.052379 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a47d54-0db2-47b0-94af-83ad2fde666c None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:58.052767 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a47d54-0db2-47b0-94af-83ad2fde666c None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:58.052961 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a47d54-0db2-47b0-94af-83ad2fde666c None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:58.053656 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a47d54-0db2-47b0-94af-83ad2fde666c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:58.053952 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a47d54-0db2-47b0-94af-83ad2fde666c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:58.055476 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a47d54-0db2-47b0-94af-83ad2fde666c None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:58.062446 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e41a2d-cad8-491f-8200-adb7464cb47b None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:58.068970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 247/501] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:58.070206 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9973257-29e9-49a0-921a-bfc5fa07b472 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:58.070484 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9973257-29e9-49a0-921a-bfc5fa07b472 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:58.070753 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9973257-29e9-49a0-921a-bfc5fa07b472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:58.089140 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-e9973257-29e9-49a0-921a-bfc5fa07b472 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:58.090313 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-e9973257-29e9-49a0-921a-bfc5fa07b472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:58.097465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a830c-bd62-450d-8b96-32e1f14a82af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:58.098106 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a830c-bd62-450d-8b96-32e1f14a82af None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:58.098395 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a830c-bd62-450d-8b96-32e1f14a82af None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:58.098671 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a830c-bd62-450d-8b96-32e1f14a82af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:58.110686 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a830c-bd62-450d-8b96-32e1f14a82af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:58.112193 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a830c-bd62-450d-8b96-32e1f14a82af None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:58.128424 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-d91a830c-bd62-450d-8b96-32e1f14a82af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:58.128424 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:58.128424 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:58.128424 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:58.128424 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:58.128424 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:58.128424 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:58.128424 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:58.128424 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:58.128424 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:58.128424 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:58.128424 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:58.128424 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:58.128424 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:58.128424 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:58.129505 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:58.129505 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:58.129505 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:58.129505 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:58.129505 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:58.129505 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:58.129505 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:58.129505 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:58.129505 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:58.129505 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:58.129505 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:58.129505 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:58.129505 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:58.129505 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:58.129505 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:58.129505 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:58.129505 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:58.130466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:58.130466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:58.130466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:58.130466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:49:58.130466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 255/502] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:58.130466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a830c-bd62-450d-8b96-32e1f14a82af None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:58.155950 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:58.168765 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 248/503] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:58.170138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9973257-29e9-49a0-921a-bfc5fa07b472 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:58.249112 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a830c-bd62-450d-8b96-32e1f14a82af None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:58.284687 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9973257-29e9-49a0-921a-bfc5fa07b472 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:58.288961 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94599af-d6df-482e-8752-4489705d2b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:58.289578 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94599af-d6df-482e-8752-4489705d2b02 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:58.289862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94599af-d6df-482e-8752-4489705d2b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:58.290140 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94599af-d6df-482e-8752-4489705d2b02 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:58.290878 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94599af-d6df-482e-8752-4489705d2b02 None admin] RBAC: Adding query filter params (name=service) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:58.291178 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94599af-d6df-482e-8752-4489705d2b02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:58.292601 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94599af-d6df-482e-8752-4489705d2b02 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:58.297595 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:58.307255 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 256/504] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Aug 5 19:49:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:58.308524 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94599af-d6df-482e-8752-4489705d2b02 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:58.327709 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca5a282-65aa-4393-9533-af1ddd1bb64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:58.328324 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5a282-65aa-4393-9533-af1ddd1bb64b None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:58.328623 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5a282-65aa-4393-9533-af1ddd1bb64b None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:58.328899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5a282-65aa-4393-9533-af1ddd1bb64b None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:58.329710 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca5a282-65aa-4393-9533-af1ddd1bb64b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:58.330011 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca5a282-65aa-4393-9533-af1ddd1bb64b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:58.331453 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca5a282-65aa-4393-9533-af1ddd1bb64b None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:58.371570 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 249/505] 10.222.0.39 () {56 vars in 1133 bytes} [Fri Aug 5 19:49:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:58.372285 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca5a282-65aa-4393-9533-af1ddd1bb64b None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:58.409587 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:58.433772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94599af-d6df-482e-8752-4489705d2b02 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:58.476326 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195554c3-6324-4667-960a-4b2ec6f2331d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:58.477262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195554c3-6324-4667-960a-4b2ec6f2331d None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:58.477649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195554c3-6324-4667-960a-4b2ec6f2331d None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:58.478022 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195554c3-6324-4667-960a-4b2ec6f2331d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:58.488165 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195554c3-6324-4667-960a-4b2ec6f2331d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:58.489695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195554c3-6324-4667-960a-4b2ec6f2331d None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:58.502286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-195554c3-6324-4667-960a-4b2ec6f2331d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 19:49:58.502286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:58.502286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:58.502286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:58.502286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:58.502286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:58.502286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:58.502286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:58.502286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:58.502286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:58.502286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:58.502286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:58.502286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:49:58.502286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:49:58.502286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:49:58.502286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:49:58.503512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:58.503512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:58.503512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:49:58.503512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:58.503512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:49:58.503512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:58.503512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:58.503512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:58.503512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:58.503512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:58.503512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:58.503512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:58.503512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:58.503512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:58.503512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:58.503512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:58.503512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:58.503512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:58.504908 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:58.504908 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:58.504908 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:49:58.504908 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:49:58.504908 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:49:58.504908 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:49:58.504908 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:49:58.504908 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:49:58.504908 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 19:49:58.504908 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:49:58.505663 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca5a282-65aa-4393-9533-af1ddd1bb64b None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:58.508222 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 257/506] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:58.508924 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195554c3-6324-4667-960a-4b2ec6f2331d None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:58.554925 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672ff83-096f-4fc3-88df-5bd3b17c3a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:58.555693 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672ff83-096f-4fc3-88df-5bd3b17c3a72 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:58.556044 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672ff83-096f-4fc3-88df-5bd3b17c3a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:58.556328 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672ff83-096f-4fc3-88df-5bd3b17c3a72 None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:58.557063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3672ff83-096f-4fc3-88df-5bd3b17c3a72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:58.557370 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3672ff83-096f-4fc3-88df-5bd3b17c3a72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:58.558914 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3672ff83-096f-4fc3-88df-5bd3b17c3a72 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:58.572017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 250/507] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:58.573378 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672ff83-096f-4fc3-88df-5bd3b17c3a72 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:58.606614 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:58.632352 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195554c3-6324-4667-960a-4b2ec6f2331d None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:58.673162 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badffabf-37d3-4ef4-9b30-3cab3ab59bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:58.673798 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badffabf-37d3-4ef4-9b30-3cab3ab59bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:58.674072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badffabf-37d3-4ef4-9b30-3cab3ab59bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:58.674344 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badffabf-37d3-4ef4-9b30-3cab3ab59bc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:58.687695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badffabf-37d3-4ef4-9b30-3cab3ab59bc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:58.689337 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badffabf-37d3-4ef4-9b30-3cab3ab59bc3 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:58.691307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:58.691449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672ff83-096f-4fc3-88df-5bd3b17c3a72 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:58.705886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-badffabf-37d3-4ef4-9b30-3cab3ab59bc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:58.705886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:58.705886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:58.705886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:58.705886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:58.705886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:58.705886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:58.705886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:58.705886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:58.705886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:58.705886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:58.705886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:58.705886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:58.705886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:58.705886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:58.707762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:58.707762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:58.707762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:58.707762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:58.707762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:58.707762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:58.707762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:58.707762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:58.707762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:58.707762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:58.707762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:58.707762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:58.707762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:58.707762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:58.707762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:58.707762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:58.707762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:58.709700 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:58.709700 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:58.709700 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:58.709700 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:49:58.709700 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 258/508] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:58.709700 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badffabf-37d3-4ef4-9b30-3cab3ab59bc3 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:58.739609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7f516-2b4b-463c-aa3d-c26dd4da7673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:58.740925 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7f516-2b4b-463c-aa3d-c26dd4da7673 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:58.741174 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7f516-2b4b-463c-aa3d-c26dd4da7673 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:58.741448 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7f516-2b4b-463c-aa3d-c26dd4da7673 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:58.756304 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed7f516-2b4b-463c-aa3d-c26dd4da7673 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:58.759261 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed7f516-2b4b-463c-aa3d-c26dd4da7673 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:58.782017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-5ed7f516-2b4b-463c-aa3d-c26dd4da7673 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:58.782017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:58.782017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:58.782017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:58.782017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:58.782017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:58.782017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:58.782017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:58.782017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:58.782017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:58.782017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:58.782017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:58.782017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:58.782017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:58.782017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:58.783092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:58.783092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:58.783092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:58.783092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:58.783092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:58.783092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:58.783092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:58.783092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:58.783092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:58.783092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:58.783092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:58.783092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:58.783092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:58.783092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:58.783092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:58.783092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:58.783092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:58.789452 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:58.789452 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:58.789452 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:58.789452 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:58.789452 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 251/509] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:58.789452 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7f516-2b4b-463c-aa3d-c26dd4da7673 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:58.826786 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badffabf-37d3-4ef4-9b30-3cab3ab59bc3 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:58.874656 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cb32fc-dd0c-44b8-8768-92c6fcd39885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:58.875368 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb32fc-dd0c-44b8-8768-92c6fcd39885 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:58.875649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb32fc-dd0c-44b8-8768-92c6fcd39885 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:58.875919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb32fc-dd0c-44b8-8768-92c6fcd39885 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:58.883741 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:58.905968 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7f516-2b4b-463c-aa3d-c26dd4da7673 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:58.942097 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7fc1c-b235-43a2-b40b-2a45afe6bf43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:58.942961 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7fc1c-b235-43a2-b40b-2a45afe6bf43 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:58.943292 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7fc1c-b235-43a2-b40b-2a45afe6bf43 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:58.943628 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7fc1c-b235-43a2-b40b-2a45afe6bf43 None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:58.944638 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b7fc1c-b235-43a2-b40b-2a45afe6bf43 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:58.944999 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:58.945164 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b7fc1c-b235-43a2-b40b-2a45afe6bf43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:58.946828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b7fc1c-b235-43a2-b40b-2a45afe6bf43 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:58.954242 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cb32fc-dd0c-44b8-8768-92c6fcd39885 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:58.955704 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cb32fc-dd0c-44b8-8768-92c6fcd39885 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:58.966939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 252/510] 10.222.0.39 () {56 vars in 1137 bytes} [Fri Aug 5 19:49:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:58.967896 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7fc1c-b235-43a2-b40b-2a45afe6bf43 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:59.060604 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 259/511] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:59.061670 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cb32fc-dd0c-44b8-8768-92c6fcd39885 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:59.084639 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7fc1c-b235-43a2-b40b-2a45afe6bf43 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:59.124014 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c635e424-da25-4b6f-9c03-c42f55048673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:59.124775 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635e424-da25-4b6f-9c03-c42f55048673 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:59.125089 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635e424-da25-4b6f-9c03-c42f55048673 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:59.125384 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635e424-da25-4b6f-9c03-c42f55048673 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:59.134596 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c635e424-da25-4b6f-9c03-c42f55048673 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:59.136474 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c635e424-da25-4b6f-9c03-c42f55048673 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:59.139588 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:59.154426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-c635e424-da25-4b6f-9c03-c42f55048673 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 05 19:49:59.154426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:59.154426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:59.154426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:59.154426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:59.154426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:59.154426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:59.154426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:59.154426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:59.154426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:59.154426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:59.154426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:59.154426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:49:59.154426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:49:59.154426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:49:59.154426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:49:59.156142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:59.156142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:59.156142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:49:59.156142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:59.156142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:49:59.156142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:59.156142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:59.156142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:59.156142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:59.156142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:59.156142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:59.156142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:59.156142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:59.156142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:59.156142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:59.156142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:59.156142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:59.156142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:59.158496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:59.158496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:59.158496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:49:59.158496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:49:59.158496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:49:59.158496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:49:59.158496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:49:59.158496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:49:59.158496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 05 19:49:59.158496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:59.158496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 253/512] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:58 2022] GET /identity/v3/users/glance => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:59.158496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c635e424-da25-4b6f-9c03-c42f55048673 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:59.184201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:59.197299 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cb32fc-dd0c-44b8-8768-92c6fcd39885 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:59.242490 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da78af-10c2-4a9d-a5a1-00cdd54e82ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:59.243103 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da78af-10c2-4a9d-a5a1-00cdd54e82ac None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:59.243387 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da78af-10c2-4a9d-a5a1-00cdd54e82ac None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:59.243673 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da78af-10c2-4a9d-a5a1-00cdd54e82ac None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:59.244379 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da78af-10c2-4a9d-a5a1-00cdd54e82ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:59.244716 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da78af-10c2-4a9d-a5a1-00cdd54e82ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:59.246124 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da78af-10c2-4a9d-a5a1-00cdd54e82ac None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:59.258401 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 260/513] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:59.259327 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da78af-10c2-4a9d-a5a1-00cdd54e82ac None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:59.301357 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c635e424-da25-4b6f-9c03-c42f55048673 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:59.344677 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48bc716-a8d4-4af2-9929-538d3281470b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:59.345250 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48bc716-a8d4-4af2-9929-538d3281470b None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:59.345622 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48bc716-a8d4-4af2-9929-538d3281470b None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:59.345994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48bc716-a8d4-4af2-9929-538d3281470b None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:59.347045 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48bc716-a8d4-4af2-9929-538d3281470b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:59.347685 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48bc716-a8d4-4af2-9929-538d3281470b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:59.349284 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48bc716-a8d4-4af2-9929-538d3281470b None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:59.366588 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 254/514] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:59.367499 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48bc716-a8d4-4af2-9929-538d3281470b None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:59.393577 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da78af-10c2-4a9d-a5a1-00cdd54e82ac None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:59.434179 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbdb1b3-d5d2-4cb8-9e80-424cf7c06308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:59.435527 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdb1b3-d5d2-4cb8-9e80-424cf7c06308 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:59.435527 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdb1b3-d5d2-4cb8-9e80-424cf7c06308 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:59.435823 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdb1b3-d5d2-4cb8-9e80-424cf7c06308 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:59.444662 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:59.470815 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:59.496615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48bc716-a8d4-4af2-9929-538d3281470b None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:59.530366 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbdb1b3-d5d2-4cb8-9e80-424cf7c06308 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:59.532170 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbdb1b3-d5d2-4cb8-9e80-424cf7c06308 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:59.553615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f727d-a514-4205-ad98-1d13e7f4d2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:59.554463 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f727d-a514-4205-ad98-1d13e7f4d2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:59.554821 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f727d-a514-4205-ad98-1d13e7f4d2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:59.555155 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f727d-a514-4205-ad98-1d13e7f4d2a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:59.571935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16f727d-a514-4205-ad98-1d13e7f4d2a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:59.573980 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16f727d-a514-4205-ad98-1d13e7f4d2a4 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:59.592835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-e16f727d-a514-4205-ad98-1d13e7f4d2a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:59.592835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:59.592835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:59.592835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:59.592835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:59.592835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:59.592835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:59.592835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:59.592835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:59.592835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:59.592835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:59.592835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:59.592835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:59.592835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:59.592835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:59.593819 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:59.593819 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:59.593819 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:59.593819 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:59.593819 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:59.593819 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:59.593819 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:59.593819 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:59.593819 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:59.593819 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:59.593819 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:59.593819 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:59.593819 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:59.593819 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:59.593819 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:59.593819 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:59.593819 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:59.595212 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:59.595212 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:59.595212 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:59.595212 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:49:59.595212 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 255/515] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:59.595212 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f727d-a514-4205-ad98-1d13e7f4d2a4 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:59.666497 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 261/516] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:59.667587 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbdb1b3-d5d2-4cb8-9e80-424cf7c06308 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:59.746947 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:59.757745 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f727d-a514-4205-ad98-1d13e7f4d2a4 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:59.777901 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:59.812060 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85a6a6-7d3e-4034-9f56-3e025292ab62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:59.812992 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85a6a6-7d3e-4034-9f56-3e025292ab62 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:59.813410 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85a6a6-7d3e-4034-9f56-3e025292ab62 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:59.813798 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbdb1b3-d5d2-4cb8-9e80-424cf7c06308 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:59.814206 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85a6a6-7d3e-4034-9f56-3e025292ab62 None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:59.815151 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c85a6a6-7d3e-4034-9f56-3e025292ab62 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:59.815667 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c85a6a6-7d3e-4034-9f56-3e025292ab62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:59.817939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c85a6a6-7d3e-4034-9f56-3e025292ab62 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:59.844942 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 256/517] 10.222.0.39 () {56 vars in 1137 bytes} [Fri Aug 5 19:49:59 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:59.845823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85a6a6-7d3e-4034-9f56-3e025292ab62 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:59.864340 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb238f-0458-4536-bab4-8bece28e79fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:59.864988 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb238f-0458-4536-bab4-8bece28e79fb None admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:59.865349 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb238f-0458-4536-bab4-8bece28e79fb None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:59.865717 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb238f-0458-4536-bab4-8bece28e79fb None admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:59.866507 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb238f-0458-4536-bab4-8bece28e79fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:59.868276 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb238f-0458-4536-bab4-8bece28e79fb None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:59.893765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 262/518] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Aug 5 19:49:59 2022] POST /identity/v3/users => generated 247 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:59.894631 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb238f-0458-4536-bab4-8bece28e79fb None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:00.003193 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85a6a6-7d3e-4034-9f56-3e025292ab62 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:00.048828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e71ceb-2b3c-462f-8735-0640d44c365c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:00.049626 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e71ceb-2b3c-462f-8735-0640d44c365c None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:00.049908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e71ceb-2b3c-462f-8735-0640d44c365c None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:00.050229 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e71ceb-2b3c-462f-8735-0640d44c365c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:00.062047 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e71ceb-2b3c-462f-8735-0640d44c365c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:00.064490 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e71ceb-2b3c-462f-8735-0640d44c365c None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:00.079693 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-25e71ceb-2b3c-462f-8735-0640d44c365c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 19:50:00.079693 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:00.079693 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:00.079693 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:00.079693 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:00.079693 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:00.079693 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:00.079693 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:00.079693 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:00.079693 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:00.079693 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:00.079693 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:00.079693 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:00.079693 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:00.079693 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:00.079693 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:00.081020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:00.081020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:00.081020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:00.081020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:00.081020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:00.081020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:00.081020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:00.081020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:00.081020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:00.081020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:00.081020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:00.081020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:00.081020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:00.081020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:00.081020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:00.081020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:00.081020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:00.081020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:00.082025 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:00.082025 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:00.082025 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:00.082025 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:00.082025 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:00.082025 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:00.082025 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:00.082025 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:00.082025 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 19:50:00.082025 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:00.082544 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb238f-0458-4536-bab4-8bece28e79fb None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:00.082625 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 257/519] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Aug 5 19:49:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:00.082625 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e71ceb-2b3c-462f-8735-0640d44c365c None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:00.101707 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:00.138788 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:00.155354 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e369a-f4cc-4c86-9312-a70d314d1bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:00.156210 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e369a-f4cc-4c86-9312-a70d314d1bcd None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:00.157362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e369a-f4cc-4c86-9312-a70d314d1bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:00.158199 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e369a-f4cc-4c86-9312-a70d314d1bcd None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:00.159684 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e369a-f4cc-4c86-9312-a70d314d1bcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:00.161104 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e369a-f4cc-4c86-9312-a70d314d1bcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:00.163131 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e369a-f4cc-4c86-9312-a70d314d1bcd None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:00.182349 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 263/520] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:00.183633 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e369a-f4cc-4c86-9312-a70d314d1bcd None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:00.260843 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e71ceb-2b3c-462f-8735-0640d44c365c None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:00.309003 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29dfdb-2eb6-48cc-9f68-6c1254dbb5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:00.310057 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29dfdb-2eb6-48cc-9f68-6c1254dbb5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:00.310240 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29dfdb-2eb6-48cc-9f68-6c1254dbb5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:00.310555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29dfdb-2eb6-48cc-9f68-6c1254dbb5f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:00.323542 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29dfdb-2eb6-48cc-9f68-6c1254dbb5f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:00.326079 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29dfdb-2eb6-48cc-9f68-6c1254dbb5f2 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:00.339703 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e369a-f4cc-4c86-9312-a70d314d1bcd None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:00.346483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-3a29dfdb-2eb6-48cc-9f68-6c1254dbb5f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:00.346483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:00.346483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:00.346483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:00.346483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:00.346483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:00.346483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:00.346483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:00.346483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:00.346483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:00.346483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:00.346483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:00.346483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:00.346483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:00.346483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:00.355238 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:00.355238 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:00.355238 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:00.355238 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:00.355238 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:00.355238 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:00.355238 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:00.355238 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:00.355238 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:00.355238 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:00.355238 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:00.355238 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:00.355238 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:00.355238 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:00.355238 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:00.355238 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:00.355238 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:00.356488 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:00.356488 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:00.356488 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:00.356488 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:00.356488 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 258/521] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:00.356488 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29dfdb-2eb6-48cc-9f68-6c1254dbb5f2 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:00.407889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61dde03-306d-4b0d-b44a-feca1f440e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:00.408874 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61dde03-306d-4b0d-b44a-feca1f440e6c None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:00.409033 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61dde03-306d-4b0d-b44a-feca1f440e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:00.409333 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61dde03-306d-4b0d-b44a-feca1f440e6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:00.479845 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:00.502698 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:00.524748 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61dde03-306d-4b0d-b44a-feca1f440e6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:00.526209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61dde03-306d-4b0d-b44a-feca1f440e6c None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:00.537502 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29dfdb-2eb6-48cc-9f68-6c1254dbb5f2 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:00.605369 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb67f7-1ac3-4a69-a0b7-abd9706eaeb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:00.606128 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb67f7-1ac3-4a69-a0b7-abd9706eaeb7 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:00.606475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb67f7-1ac3-4a69-a0b7-abd9706eaeb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:00.607426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb67f7-1ac3-4a69-a0b7-abd9706eaeb7 None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:00.608293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb67f7-1ac3-4a69-a0b7-abd9706eaeb7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:00.608630 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb67f7-1ac3-4a69-a0b7-abd9706eaeb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:00.610635 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb67f7-1ac3-4a69-a0b7-abd9706eaeb7 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:00.640480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 259/522] 10.222.0.39 () {56 vars in 1139 bytes} [Fri Aug 5 19:50:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:00.641517 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb67f7-1ac3-4a69-a0b7-abd9706eaeb7 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:00.679510 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 264/523] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:50:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:50:00.681995 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61dde03-306d-4b0d-b44a-feca1f440e6c None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:00.780529 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb67f7-1ac3-4a69-a0b7-abd9706eaeb7 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:00.788536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:00.826169 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c8d99a-49f9-4a5a-a609-89309dca1ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:00.826822 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8d99a-49f9-4a5a-a609-89309dca1ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:00.827324 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8d99a-49f9-4a5a-a609-89309dca1ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:00.827449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8d99a-49f9-4a5a-a609-89309dca1ef1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:00.835615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c8d99a-49f9-4a5a-a609-89309dca1ef1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:00.836040 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61dde03-306d-4b0d-b44a-feca1f440e6c None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:00.837219 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c8d99a-49f9-4a5a-a609-89309dca1ef1 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:00.840283 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:00.855775 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-81c8d99a-49f9-4a5a-a609-89309dca1ef1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:00.855775 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:00.855775 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:00.855775 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:00.855775 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:00.855775 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:00.855775 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:00.855775 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:00.855775 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:00.855775 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:00.855775 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:00.855775 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:00.855775 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:00.855775 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:00.855775 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:00.860066 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:00.860066 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:00.860066 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:00.860066 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:00.860066 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:00.860066 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:00.860066 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:00.860066 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:00.860066 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:00.860066 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:00.860066 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:00.860066 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:00.860066 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:00.860066 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:00.860066 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:00.860066 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:00.860066 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:00.860984 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:00.860984 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:00.860984 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:00.860984 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:00.860984 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:00.860984 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:00.860984 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:00.860984 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:00.862740 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 260/524] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:00.863736 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c8d99a-49f9-4a5a-a609-89309dca1ef1 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:00.902286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b3fe0-6c77-4be1-8cdd-a30231f6b216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:00.903061 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b3fe0-6c77-4be1-8cdd-a30231f6b216 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:00.903379 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b3fe0-6c77-4be1-8cdd-a30231f6b216 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:00.904700 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b3fe0-6c77-4be1-8cdd-a30231f6b216 None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:00.905118 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b3fe0-6c77-4be1-8cdd-a30231f6b216 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:00.905436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b3fe0-6c77-4be1-8cdd-a30231f6b216 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:00.906888 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b3fe0-6c77-4be1-8cdd-a30231f6b216 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:00.926821 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 265/525] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:00.926821 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b3fe0-6c77-4be1-8cdd-a30231f6b216 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:01.028193 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c8d99a-49f9-4a5a-a609-89309dca1ef1 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:01.068925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b3fe0-6c77-4be1-8cdd-a30231f6b216 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:01.092532 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2d8d6d-1701-4809-b03c-e1747fff829e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:01.093183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d8d6d-1701-4809-b03c-e1747fff829e None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:01.093459 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d8d6d-1701-4809-b03c-e1747fff829e None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:01.093793 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d8d6d-1701-4809-b03c-e1747fff829e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:01.114325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:01.123968 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf13c0b-063b-427f-bb73-2958691fb6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:01.124702 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf13c0b-063b-427f-bb73-2958691fb6b1 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:01.124986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf13c0b-063b-427f-bb73-2958691fb6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:01.125368 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf13c0b-063b-427f-bb73-2958691fb6b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:01.142661 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf13c0b-063b-427f-bb73-2958691fb6b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:01.144534 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf13c0b-063b-427f-bb73-2958691fb6b1 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:01.165383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-2cf13c0b-063b-427f-bb73-2958691fb6b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:01.165383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:01.165383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:01.165383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:01.165383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:01.165383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:01.165383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:01.165383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:01.165383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:01.165383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:01.165383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:01.165383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:01.165383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:01.165383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:01.165383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:01.166382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:01.166382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:01.166382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:01.166382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:01.166382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:01.166382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:01.166382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:01.166382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:01.166382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:01.166382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:01.166382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:01.166382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:01.166382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:01.166382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:01.166382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:01.166382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:01.166382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:01.167279 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:01.167279 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:01.167279 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:01.167279 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:01.173040 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 266/526] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:01.174528 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf13c0b-063b-427f-bb73-2958691fb6b1 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:01.208141 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:01.213026 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2d8d6d-1701-4809-b03c-e1747fff829e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:01.215181 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2d8d6d-1701-4809-b03c-e1747fff829e None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:01.324112 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf13c0b-063b-427f-bb73-2958691fb6b1 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:01.367870 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e392db97-c980-4b90-882f-c76f44122aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:01.368773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392db97-c980-4b90-882f-c76f44122aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:01.369149 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392db97-c980-4b90-882f-c76f44122aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:01.369315 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392db97-c980-4b90-882f-c76f44122aa2 None admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:01.370399 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e392db97-c980-4b90-882f-c76f44122aa2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:01.370731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e392db97-c980-4b90-882f-c76f44122aa2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:01.372135 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e392db97-c980-4b90-882f-c76f44122aa2 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:01.378490 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 261/527] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:50:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:50:01.378490 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2d8d6d-1701-4809-b03c-e1747fff829e None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:01.404627 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 267/528] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:01.404627 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e392db97-c980-4b90-882f-c76f44122aa2 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:01.478792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:01.507699 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:01.515769 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2d8d6d-1701-4809-b03c-e1747fff829e None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:01.562356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e392db97-c980-4b90-882f-c76f44122aa2 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:01.563087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471950b7-3da1-48bb-bf44-ef27670b5a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:01.563267 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471950b7-3da1-48bb-bf44-ef27670b5a10 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:01.563568 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471950b7-3da1-48bb-bf44-ef27670b5a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:01.563828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471950b7-3da1-48bb-bf44-ef27670b5a10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:01.572496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471950b7-3da1-48bb-bf44-ef27670b5a10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:01.574036 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471950b7-3da1-48bb-bf44-ef27670b5a10 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:01.587197 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-471950b7-3da1-48bb-bf44-ef27670b5a10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:01.587197 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:01.587197 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:01.587197 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:01.587197 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:01.587197 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:01.587197 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:01.587197 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:01.587197 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:01.587197 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:01.587197 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:01.587197 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:01.587197 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:01.587197 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:01.587197 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:01.588083 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:01.588083 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:01.588083 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:01.588083 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:01.588083 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:01.588083 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:01.588083 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:01.588083 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:01.588083 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:01.588083 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:01.588083 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:01.588083 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:01.588083 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:01.588083 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:01.588083 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:01.588083 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:01.588083 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:01.589582 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:01.589582 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:01.589582 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:01.589582 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:01.589582 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:01.589582 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:01.589582 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:01.589582 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:01.589582 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 262/529] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:01.590080 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471950b7-3da1-48bb-bf44-ef27670b5a10 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:01.636202 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d626c-3e1b-4e65-8ece-869e932b1434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:01.637211 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d626c-3e1b-4e65-8ece-869e932b1434 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:01.637645 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d626c-3e1b-4e65-8ece-869e932b1434 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:01.638621 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d626c-3e1b-4e65-8ece-869e932b1434 None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:01.639721 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d626c-3e1b-4e65-8ece-869e932b1434 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:01.640380 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d626c-3e1b-4e65-8ece-869e932b1434 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:01.642674 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d626c-3e1b-4e65-8ece-869e932b1434 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:01.659567 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 268/530] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:01.660472 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d626c-3e1b-4e65-8ece-869e932b1434 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:01.743837 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471950b7-3da1-48bb-bf44-ef27670b5a10 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:01.790387 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156268fd-8206-498c-b2ba-6d62a64c56b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:01.791755 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156268fd-8206-498c-b2ba-6d62a64c56b6 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:01.792053 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156268fd-8206-498c-b2ba-6d62a64c56b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:01.792737 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156268fd-8206-498c-b2ba-6d62a64c56b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:01.798711 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d626c-3e1b-4e65-8ece-869e932b1434 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:01.800878 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:01.809263 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156268fd-8206-498c-b2ba-6d62a64c56b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:01.812315 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156268fd-8206-498c-b2ba-6d62a64c56b6 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:01.831894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-156268fd-8206-498c-b2ba-6d62a64c56b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:01.831894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:01.831894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:01.831894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:01.831894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:01.831894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:01.831894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:01.831894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:01.831894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:01.831894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:01.831894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:01.831894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:01.831894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:01.831894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:01.831894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:01.833353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:01.833353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:01.833353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:01.833353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:01.833353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:01.833353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:01.833353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:01.833353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:01.833353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:01.833353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:01.833353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:01.833353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:01.833353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:01.833353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:01.833353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:01.833353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:01.833353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:01.834491 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:01.834491 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:01.834491 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:01.834491 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:01.836791 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 263/531] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:01.837922 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156268fd-8206-498c-b2ba-6d62a64c56b6 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:01.849636 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547467b3-7dff-4aca-b5c2-c10218b85a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:01.850376 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547467b3-7dff-4aca-b5c2-c10218b85a8b None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:01.850607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547467b3-7dff-4aca-b5c2-c10218b85a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:01.850975 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547467b3-7dff-4aca-b5c2-c10218b85a8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:01.851909 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547467b3-7dff-4aca-b5c2-c10218b85a8b None admin] RBAC: Adding query filter params (role.id=7ebfd9e58fe549f08cdaec5f906a38a6, scope.project.id=80379e2104644e98a2e85d4e4af1b365, user.id=95e9f085e4f04d118373b8abff27543a) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:01.852306 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547467b3-7dff-4aca-b5c2-c10218b85a8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:01.853797 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:01.854374 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547467b3-7dff-4aca-b5c2-c10218b85a8b None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:01.872981 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 269/532] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:50:01 2022] GET /identity/v3/role_assignments?user.id=95e9f085e4f04d118373b8abff27543a&scope.project.id=80379e2104644e98a2e85d4e4af1b365&role.id=7ebfd9e58fe549f08cdaec5f906a38a6 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:01.872981 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547467b3-7dff-4aca-b5c2-c10218b85a8b None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:01.995458 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156268fd-8206-498c-b2ba-6d62a64c56b6 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:02.027349 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547467b3-7dff-4aca-b5c2-c10218b85a8b None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:02.037322 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49af6e-0dcf-436a-96a1-daede8da9bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:02.038148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49af6e-0dcf-436a-96a1-daede8da9bea None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:02.039770 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49af6e-0dcf-436a-96a1-daede8da9bea None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:02.039770 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49af6e-0dcf-436a-96a1-daede8da9bea None admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:02.039770 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49af6e-0dcf-436a-96a1-daede8da9bea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:02.040442 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49af6e-0dcf-436a-96a1-daede8da9bea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:02.043014 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49af6e-0dcf-436a-96a1-daede8da9bea None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:02.067412 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 264/533] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:02.068414 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49af6e-0dcf-436a-96a1-daede8da9bea None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:02.078708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8427ce03-4f14-4530-91f8-89f29dab9825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:02.079685 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427ce03-4f14-4530-91f8-89f29dab9825 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:02.080193 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427ce03-4f14-4530-91f8-89f29dab9825 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:02.080822 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427ce03-4f14-4530-91f8-89f29dab9825 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:02.089514 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8427ce03-4f14-4530-91f8-89f29dab9825 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:02.091025 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8427ce03-4f14-4530-91f8-89f29dab9825 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:02.105764 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-8427ce03-4f14-4530-91f8-89f29dab9825 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:02.105764 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:02.105764 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:02.105764 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:02.105764 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:02.105764 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:02.105764 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:02.105764 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:02.105764 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:02.105764 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:02.105764 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:02.105764 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:02.105764 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:02.105764 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:02.105764 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:02.107459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:02.107459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:02.107459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:02.107459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:02.107459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:02.107459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:02.107459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:02.107459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:02.107459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:02.107459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:02.107459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:02.107459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:02.107459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:02.107459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:02.107459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:02.107459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:02.107459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:02.108708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:02.108708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:02.108708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:02.108708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:02.108708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:02.108708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:02.108708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:02.108708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:02.115125 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 270/534] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:02.115125 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8427ce03-4f14-4530-91f8-89f29dab9825 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:02.158642 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:02.177673 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:02.225695 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49af6e-0dcf-436a-96a1-daede8da9bea None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:02.248925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8427ce03-4f14-4530-91f8-89f29dab9825 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:02.276762 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae28c20d-2e0c-4087-8f2d-0968c986d019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:02.277793 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28c20d-2e0c-4087-8f2d-0968c986d019 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:02.278325 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28c20d-2e0c-4087-8f2d-0968c986d019 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:02.279381 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28c20d-2e0c-4087-8f2d-0968c986d019 None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:02.280538 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae28c20d-2e0c-4087-8f2d-0968c986d019 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:02.281091 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae28c20d-2e0c-4087-8f2d-0968c986d019 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:02.283268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae28c20d-2e0c-4087-8f2d-0968c986d019 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:02.284936 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d1e52b-9148-494c-80a9-1927b9c9283a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:02.285554 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d1e52b-9148-494c-80a9-1927b9c9283a None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:02.285834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d1e52b-9148-494c-80a9-1927b9c9283a None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:02.286111 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d1e52b-9148-494c-80a9-1927b9c9283a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:02.286910 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d1e52b-9148-494c-80a9-1927b9c9283a None admin] RBAC: Adding query filter params (role.id=7ebfd9e58fe549f08cdaec5f906a38a6, scope.project.id=80379e2104644e98a2e85d4e4af1b365, user.id=44dfe45766d743558ed82e464721cf72) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:02.287224 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d1e52b-9148-494c-80a9-1927b9c9283a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:02.288673 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d1e52b-9148-494c-80a9-1927b9c9283a None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:02.294621 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 271/535] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:50:02 2022] GET /identity/v3/role_assignments?user.id=44dfe45766d743558ed82e464721cf72&scope.project.id=80379e2104644e98a2e85d4e4af1b365&role.id=7ebfd9e58fe549f08cdaec5f906a38a6 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:02.295602 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d1e52b-9148-494c-80a9-1927b9c9283a None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:02.308133 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 265/536] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:02.310990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902202b6-6f91-458a-b298-18a3ef1d14d4 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:02.311551 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902202b6-6f91-458a-b298-18a3ef1d14d4 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:02.312131 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902202b6-6f91-458a-b298-18a3ef1d14d4 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:02.319284 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 266/537] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:02.319284 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902202b6-6f91-458a-b298-18a3ef1d14d4 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:02.421782 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:02.431130 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d1e52b-9148-494c-80a9-1927b9c9283a None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:02.480607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610bffa5-0e21-4989-af0b-62b7fd8a3425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:02.480607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610bffa5-0e21-4989-af0b-62b7fd8a3425 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:02.480607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610bffa5-0e21-4989-af0b-62b7fd8a3425 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:02.480607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610bffa5-0e21-4989-af0b-62b7fd8a3425 None admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:02.481035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610bffa5-0e21-4989-af0b-62b7fd8a3425 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:02.481035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610bffa5-0e21-4989-af0b-62b7fd8a3425 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:02.481035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610bffa5-0e21-4989-af0b-62b7fd8a3425 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:02.485341 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902202b6-6f91-458a-b298-18a3ef1d14d4 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:02.494351 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 272/538] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:02.496134 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d95bea-4c70-4672-b2df-a7a3e6412e90 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:02.496424 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d95bea-4c70-4672-b2df-a7a3e6412e90 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:02.496831 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d95bea-4c70-4672-b2df-a7a3e6412e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:02.520720 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-a0d95bea-4c70-4672-b2df-a7a3e6412e90 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:02.522216 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-a0d95bea-4c70-4672-b2df-a7a3e6412e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:02.523331 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:02.531841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0294827-174d-4f94-a9a5-c64477a31718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:02.533385 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0294827-174d-4f94-a9a5-c64477a31718 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:02.533874 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0294827-174d-4f94-a9a5-c64477a31718 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:02.534345 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0294827-174d-4f94-a9a5-c64477a31718 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:02.546017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0294827-174d-4f94-a9a5-c64477a31718 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:02.547889 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0294827-174d-4f94-a9a5-c64477a31718 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:02.566679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-d0294827-174d-4f94-a9a5-c64477a31718 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:02.566679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:02.566679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:02.566679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:02.566679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:02.566679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:02.566679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:02.566679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:02.566679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:02.566679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:02.566679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:02.566679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:02.566679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:02.566679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:02.566679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:02.567601 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:02.567601 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:02.567601 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:02.567601 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:02.567601 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:02.567601 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:02.567601 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:02.567601 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:02.567601 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:02.567601 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:02.567601 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:02.567601 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:02.567601 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:02.567601 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:02.567601 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:02.567601 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:02.567601 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:02.568458 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:02.568458 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:02.568458 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:02.568458 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:02.568458 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:02.568458 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:02.568458 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:02.568458 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:02.568903 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 267/539] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:02.569631 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0294827-174d-4f94-a9a5-c64477a31718 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:02.624729 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 273/540] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:50:02.626810 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d95bea-4c70-4672-b2df-a7a3e6412e90 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:02.703085 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0294827-174d-4f94-a9a5-c64477a31718 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:02.748364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f2369-23ef-4117-8f70-3ddd50f68aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:02.748799 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f2369-23ef-4117-8f70-3ddd50f68aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:02.749114 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f2369-23ef-4117-8f70-3ddd50f68aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:02.749235 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f2369-23ef-4117-8f70-3ddd50f68aa2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:02.750038 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5f2369-23ef-4117-8f70-3ddd50f68aa2 None admin] RBAC: Adding query filter params (role.id=7ebfd9e58fe549f08cdaec5f906a38a6, scope.project.id=80379e2104644e98a2e85d4e4af1b365, user.id=2fe69b398c1f4e18b625a63ae40b01e8) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:02.750422 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5f2369-23ef-4117-8f70-3ddd50f68aa2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:02.751998 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5f2369-23ef-4117-8f70-3ddd50f68aa2 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:02.753496 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:02.759893 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 268/541] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:50:02 2022] GET /identity/v3/role_assignments?user.id=2fe69b398c1f4e18b625a63ae40b01e8&scope.project.id=80379e2104644e98a2e85d4e4af1b365&role.id=7ebfd9e58fe549f08cdaec5f906a38a6 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:02.760853 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f2369-23ef-4117-8f70-3ddd50f68aa2 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:02.780540 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d95bea-4c70-4672-b2df-a7a3e6412e90 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:02.824759 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fca171-ff14-4007-bb9a-b0a2b2f3c8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:02.827146 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fca171-ff14-4007-bb9a-b0a2b2f3c8c7 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:02.827753 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fca171-ff14-4007-bb9a-b0a2b2f3c8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:02.828229 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fca171-ff14-4007-bb9a-b0a2b2f3c8c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:02.829340 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fca171-ff14-4007-bb9a-b0a2b2f3c8c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:02.829855 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fca171-ff14-4007-bb9a-b0a2b2f3c8c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:02.831669 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fca171-ff14-4007-bb9a-b0a2b2f3c8c7 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:02.846528 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 274/542] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:02.855793 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fca171-ff14-4007-bb9a-b0a2b2f3c8c7 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:02.864962 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:02.930824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f2369-23ef-4117-8f70-3ddd50f68aa2 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:02.972433 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a3f80-e73e-4592-8655-edd27fdfcb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:02.973119 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a3f80-e73e-4592-8655-edd27fdfcb0e None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:02.973410 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a3f80-e73e-4592-8655-edd27fdfcb0e None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:02.973688 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a3f80-e73e-4592-8655-edd27fdfcb0e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:03.001557 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329a3f80-e73e-4592-8655-edd27fdfcb0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:03.011169 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-329a3f80-e73e-4592-8655-edd27fdfcb0e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:03.011169 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:03.011169 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:03.011169 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:03.011169 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:03.011169 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:03.011169 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:03.011169 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:03.011169 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:03.011169 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:03.011169 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:03.011169 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:03.011169 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:03.011169 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:03.011169 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:03.011169 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:03.012320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:03.012320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:03.012320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:03.012320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:03.012320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:03.012320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:03.012320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:03.012320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:03.012320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:03.012320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:03.012320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:03.012320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:03.012320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:03.012320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:03.012320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:03.012320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:03.013788 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:03.013788 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:03.013788 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:03.013788 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:03.013788 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:03.013788 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:03.013788 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:03.013788 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:03.013788 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:03.013788 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:03.013788 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:03.013788 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:03.021203 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 269/543] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Aug 5 19:50:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:03.024593 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fca171-ff14-4007-bb9a-b0a2b2f3c8c7 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:03.028314 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a3f80-e73e-4592-8655-edd27fdfcb0e None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:03.086588 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8dd400-a05b-4a16-8934-ade2fdfa07e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:03.087554 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8dd400-a05b-4a16-8934-ade2fdfa07e3 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:03.088029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8dd400-a05b-4a16-8934-ade2fdfa07e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:03.088491 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8dd400-a05b-4a16-8934-ade2fdfa07e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:03.089551 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8dd400-a05b-4a16-8934-ade2fdfa07e3 None admin] RBAC: Adding query filter params (role.id=7ebfd9e58fe549f08cdaec5f906a38a6, scope.project.id=80379e2104644e98a2e85d4e4af1b365, user.id=b1c75368c00e4b0cb7b3163728843479) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:03.090229 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8dd400-a05b-4a16-8934-ade2fdfa07e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:03.093055 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8dd400-a05b-4a16-8934-ade2fdfa07e3 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:03.102350 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 275/544] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:50:02 2022] GET /identity/v3/role_assignments?user.id=b1c75368c00e4b0cb7b3163728843479&scope.project.id=80379e2104644e98a2e85d4e4af1b365&role.id=7ebfd9e58fe549f08cdaec5f906a38a6 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:03.225143 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a3f80-e73e-4592-8655-edd27fdfcb0e None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:03.259616 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:03.278151 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8974256f-8e68-48fe-98e9-9bac37652356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:03.278863 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974256f-8e68-48fe-98e9-9bac37652356 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:03.279123 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974256f-8e68-48fe-98e9-9bac37652356 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:03.279844 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974256f-8e68-48fe-98e9-9bac37652356 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:03.280849 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8974256f-8e68-48fe-98e9-9bac37652356 None admin] RBAC: Adding query filter params (name=service) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:03.281603 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8974256f-8e68-48fe-98e9-9bac37652356 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:03.283007 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8974256f-8e68-48fe-98e9-9bac37652356 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:03.292428 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 270/545] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Aug 5 19:50:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:03.298952 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8dd400-a05b-4a16-8934-ade2fdfa07e3 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:03.305825 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:03.447534 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8dd400-a05b-4a16-8934-ade2fdfa07e3 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:03.488038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2bcc30-36c3-4fea-a3a6-41e20a02f093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:03.488718 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2bcc30-36c3-4fea-a3a6-41e20a02f093 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:03.489002 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2bcc30-36c3-4fea-a3a6-41e20a02f093 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:03.489275 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2bcc30-36c3-4fea-a3a6-41e20a02f093 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:03.504105 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2bcc30-36c3-4fea-a3a6-41e20a02f093 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:03.505629 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2bcc30-36c3-4fea-a3a6-41e20a02f093 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:03.529110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-af2bcc30-36c3-4fea-a3a6-41e20a02f093 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:03.529110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:03.529110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:03.529110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:03.529110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:03.529110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:03.529110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:03.529110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:03.529110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:03.529110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:03.529110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:03.529110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:03.529110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:03.529110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:03.529110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:03.530156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:03.530156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:03.530156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:03.530156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:03.530156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:03.530156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:03.530156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:03.530156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:03.530156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:03.530156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:03.530156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:03.530156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:03.530156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:03.530156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:03.530156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:03.530156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:03.530156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:03.530981 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:03.530981 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:03.530981 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:03.530981 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:03.530981 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 276/546] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:03.536208 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8974256f-8e68-48fe-98e9-9bac37652356 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:03.718923 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8974256f-8e68-48fe-98e9-9bac37652356 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:03.761232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157944c-552e-4ca9-bbc3-f1a997f4c5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:03.761885 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157944c-552e-4ca9-bbc3-f1a997f4c5a4 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:03.762164 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157944c-552e-4ca9-bbc3-f1a997f4c5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:03.762439 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157944c-552e-4ca9-bbc3-f1a997f4c5a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:03.763407 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8157944c-552e-4ca9-bbc3-f1a997f4c5a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:03.763719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8157944c-552e-4ca9-bbc3-f1a997f4c5a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:03.765125 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8157944c-552e-4ca9-bbc3-f1a997f4c5a4 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:03.778250 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 271/547] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:03.784538 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2bcc30-36c3-4fea-a3a6-41e20a02f093 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:03.885089 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:03.938957 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2bcc30-36c3-4fea-a3a6-41e20a02f093 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:03.997100 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190bba41-9fcd-48a4-841c-553528ed02f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:03.998174 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190bba41-9fcd-48a4-841c-553528ed02f7 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:03.998575 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190bba41-9fcd-48a4-841c-553528ed02f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:03.998771 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190bba41-9fcd-48a4-841c-553528ed02f7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:04.011965 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190bba41-9fcd-48a4-841c-553528ed02f7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:04.014427 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190bba41-9fcd-48a4-841c-553528ed02f7 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:04.033377 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-190bba41-9fcd-48a4-841c-553528ed02f7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:50:04.033377 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:04.033377 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:04.033377 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:04.033377 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:04.033377 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:04.033377 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:04.033377 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:04.033377 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:04.033377 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:04.033377 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:04.033377 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:04.033377 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:04.033377 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:04.033377 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:04.033377 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:04.034388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:04.034388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:04.034388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:04.034388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:04.034388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:04.034388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:04.034388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:04.034388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:04.034388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:04.034388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:04.034388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:04.034388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:04.034388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:04.034388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:04.034388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:04.034388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:04.034388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:04.034388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:04.035970 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:04.035970 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:04.035970 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:04.035970 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:04.035970 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:04.035970 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:04.035970 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:04.035970 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:04.035970 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:50:04.035970 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:04.035970 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 277/548] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:50:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:04.044799 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157944c-552e-4ca9-bbc3-f1a997f4c5a4 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:04.128886 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:04.234862 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157944c-552e-4ca9-bbc3-f1a997f4c5a4 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:04.281089 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e5db4-f433-431b-a48e-0c26138c0182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:04.281627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e5db4-f433-431b-a48e-0c26138c0182 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:04.281906 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e5db4-f433-431b-a48e-0c26138c0182 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:04.282176 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e5db4-f433-431b-a48e-0c26138c0182 None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:04.282973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e5db4-f433-431b-a48e-0c26138c0182 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:04.283276 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e5db4-f433-431b-a48e-0c26138c0182 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:04.284725 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e5db4-f433-431b-a48e-0c26138c0182 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:04.312284 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 272/549] 10.222.0.39 () {56 vars in 1135 bytes} [Fri Aug 5 19:50:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:04.318183 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190bba41-9fcd-48a4-841c-553528ed02f7 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:04.351114 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be21811-ec5d-4854-9d10-d57da922bbcf None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:04.351462 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be21811-ec5d-4854-9d10-d57da922bbcf None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:04.351737 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be21811-ec5d-4854-9d10-d57da922bbcf None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:04.353451 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 273/550] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:04.360755 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed5f66-ffad-410d-b305-f26ec44f4327 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:04.361133 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed5f66-ffad-410d-b305-f26ec44f4327 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:04.361532 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed5f66-ffad-410d-b305-f26ec44f4327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:04.395686 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-c9ed5f66-ffad-410d-b305-f26ec44f4327 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:04.397737 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-c9ed5f66-ffad-410d-b305-f26ec44f4327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:04.553335 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 274/551] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:50:04.580736 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed5f66-ffad-410d-b305-f26ec44f4327 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:04.581286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190bba41-9fcd-48a4-841c-553528ed02f7 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:04.615254 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:04.627162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:04.655484 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbf6fe-be34-4ca6-9036-9d5e296b4b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:04.656470 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbf6fe-be34-4ca6-9036-9d5e296b4b2d None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:04.656943 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbf6fe-be34-4ca6-9036-9d5e296b4b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:04.657223 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbf6fe-be34-4ca6-9036-9d5e296b4b2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:04.792087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed5f66-ffad-410d-b305-f26ec44f4327 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:04.861886 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4d59d-c051-462d-aa0d-651a282aeaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:04.862449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4d59d-c051-462d-aa0d-651a282aeaa5 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:04.862725 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4d59d-c051-462d-aa0d-651a282aeaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:04.863049 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4d59d-c051-462d-aa0d-651a282aeaa5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:04.875309 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bbf6fe-be34-4ca6-9036-9d5e296b4b2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:04.876824 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bbf6fe-be34-4ca6-9036-9d5e296b4b2d None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:04.894635 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4d59d-c051-462d-aa0d-651a282aeaa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:04.899826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-fed4d59d-c051-462d-aa0d-651a282aeaa5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:04.899826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:04.899826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:04.899826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:04.899826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:04.899826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:04.899826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:04.899826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:04.899826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:04.899826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:04.899826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:04.899826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:04.899826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:04.899826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:04.899826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:04.899826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:04.900908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:04.900908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:04.900908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:04.900908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:04.900908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:04.900908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:04.900908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:04.900908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:04.900908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:04.900908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:04.900908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:04.900908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:04.900908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:04.900908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:04.900908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:04.900908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:04.901741 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:04.901741 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:04.901741 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:04.901741 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:04.901741 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:04.901741 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:04.901741 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:04.901741 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:04.901741 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:04.901741 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:04.901741 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:04.901741 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:04.902394 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 275/552] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Aug 5 19:50:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:04.910898 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4d59d-c051-462d-aa0d-651a282aeaa5 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:05.059052 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 278/553] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:50:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 741 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:50:05.066443 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbf6fe-be34-4ca6-9036-9d5e296b4b2d None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:05.101047 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:05.105030 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:05.163528 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4d59d-c051-462d-aa0d-651a282aeaa5 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:05.214828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a797ad-73fe-45fe-9d50-c0f87bfbfd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:05.216428 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a797ad-73fe-45fe-9d50-c0f87bfbfd0b None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:05.216960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a797ad-73fe-45fe-9d50-c0f87bfbfd0b None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:05.217485 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a797ad-73fe-45fe-9d50-c0f87bfbfd0b None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:05.218571 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a797ad-73fe-45fe-9d50-c0f87bfbfd0b None admin] RBAC: Adding query filter params (name=service) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:05.219081 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a797ad-73fe-45fe-9d50-c0f87bfbfd0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:05.220738 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a797ad-73fe-45fe-9d50-c0f87bfbfd0b None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:05.236373 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 276/554] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Aug 5 19:50:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:05.238151 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1147a7-3528-4779-a57e-7ed4b514a233 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:05.238653 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1147a7-3528-4779-a57e-7ed4b514a233 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:05.239128 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1147a7-3528-4779-a57e-7ed4b514a233 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:05.241249 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 277/555] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:05.245860 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1147a7-3528-4779-a57e-7ed4b514a233 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:05.295697 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbf6fe-be34-4ca6-9036-9d5e296b4b2d None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:05.344083 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b0d5e-a47f-40a1-b5c9-7dd69a866816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:05.345167 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b0d5e-a47f-40a1-b5c9-7dd69a866816 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:05.345753 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b0d5e-a47f-40a1-b5c9-7dd69a866816 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:05.346233 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b0d5e-a47f-40a1-b5c9-7dd69a866816 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:05.359090 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524b0d5e-a47f-40a1-b5c9-7dd69a866816 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:05.361289 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524b0d5e-a47f-40a1-b5c9-7dd69a866816 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:05.379675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-524b0d5e-a47f-40a1-b5c9-7dd69a866816 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:05.379675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:05.379675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:05.379675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:05.379675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:05.379675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:05.379675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:05.379675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:05.379675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:05.379675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:05.379675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:05.379675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:05.379675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:05.379675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:05.379675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:05.380666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:05.380666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:05.380666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:05.380666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:05.380666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:05.380666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:05.380666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:05.380666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:05.380666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:05.380666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:05.380666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:05.380666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:05.380666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:05.380666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:05.380666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:05.380666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:05.380666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:05.381543 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:05.381543 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:05.381543 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:05.381543 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:05.388487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 279/556] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:05.390276 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a626c-6cd6-4cad-9a91-203cb33f2817 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:05.390768 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a626c-6cd6-4cad-9a91-203cb33f2817 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:05.391263 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a626c-6cd6-4cad-9a91-203cb33f2817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:05.400570 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1147a7-3528-4779-a57e-7ed4b514a233 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:05.412373 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-c93a626c-6cd6-4cad-9a91-203cb33f2817 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:05.414143 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-c93a626c-6cd6-4cad-9a91-203cb33f2817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:05.459059 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a59649-9263-4e66-b577-fb5545d5a48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:05.459702 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a59649-9263-4e66-b577-fb5545d5a48e None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:05.459979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a59649-9263-4e66-b577-fb5545d5a48e None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:05.460253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a59649-9263-4e66-b577-fb5545d5a48e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:05.473918 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a59649-9263-4e66-b577-fb5545d5a48e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:05.475375 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a59649-9263-4e66-b577-fb5545d5a48e None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:05.484268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:05.493539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-98a59649-9263-4e66-b577-fb5545d5a48e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:05.493539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:05.493539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:05.493539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:05.493539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:05.493539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:05.493539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:05.493539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:05.493539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:05.493539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:05.493539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:05.493539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:05.493539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:05.493539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:05.493539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:05.494970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:05.494970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:05.494970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:05.494970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:05.494970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:05.494970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:05.494970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:05.494970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:05.494970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:05.494970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:05.494970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:05.494970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:05.494970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:05.494970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:05.494970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:05.494970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:05.494970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:05.495852 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:05.495852 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:05.495852 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:05.495852 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:05.495852 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 278/557] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:05.495852 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a59649-9263-4e66-b577-fb5545d5a48e None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:05.507941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:05.520824 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 280/558] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:50:05.520824 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141d924-2593-4176-946a-8bc0c0b96d1b None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:05.520824 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141d924-2593-4176-946a-8bc0c0b96d1b None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:05.520824 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141d924-2593-4176-946a-8bc0c0b96d1b None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:05.520824 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 281/559] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:05.521559 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141d924-2593-4176-946a-8bc0c0b96d1b None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:05.631733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a59649-9263-4e66-b577-fb5545d5a48e None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:05.664511 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141d924-2593-4176-946a-8bc0c0b96d1b None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:05.687437 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ef2cd-4730-418a-aa3b-470cc7160322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:05.687437 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ef2cd-4730-418a-aa3b-470cc7160322 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:05.687437 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ef2cd-4730-418a-aa3b-470cc7160322 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:05.687437 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ef2cd-4730-418a-aa3b-470cc7160322 None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:05.687880 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ef2cd-4730-418a-aa3b-470cc7160322 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:05.687880 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ef2cd-4730-418a-aa3b-470cc7160322 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:05.687880 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ef2cd-4730-418a-aa3b-470cc7160322 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:05.706674 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 279/560] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:05.708260 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba1f97-59a1-4dfa-b7d9-7586666db81d None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:05.708563 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba1f97-59a1-4dfa-b7d9-7586666db81d None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:05.708851 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba1f97-59a1-4dfa-b7d9-7586666db81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:05.718945 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fd1a7-b9b8-44df-8f9a-c316d34d37a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:05.719906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fd1a7-b9b8-44df-8f9a-c316d34d37a8 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:05.720360 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fd1a7-b9b8-44df-8f9a-c316d34d37a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:05.720851 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fd1a7-b9b8-44df-8f9a-c316d34d37a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:05.722183 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fd1a7-b9b8-44df-8f9a-c316d34d37a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:05.722522 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fd1a7-b9b8-44df-8f9a-c316d34d37a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:05.726318 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fd1a7-b9b8-44df-8f9a-c316d34d37a8 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:05.738482 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-09ba1f97-59a1-4dfa-b7d9-7586666db81d None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:05.741344 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-09ba1f97-59a1-4dfa-b7d9-7586666db81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:05.750738 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 282/561] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:05.750738 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fd1a7-b9b8-44df-8f9a-c316d34d37a8 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:05.825678 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:05.838869 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:05.863979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 280/562] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:50:05.865222 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba1f97-59a1-4dfa-b7d9-7586666db81d None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:05.884304 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fd1a7-b9b8-44df-8f9a-c316d34d37a8 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:05.939840 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd61083d-04d0-43f8-8ff7-a4a5067f1beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:05.940773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61083d-04d0-43f8-8ff7-a4a5067f1beb None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:05.941059 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61083d-04d0-43f8-8ff7-a4a5067f1beb None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:05.941333 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61083d-04d0-43f8-8ff7-a4a5067f1beb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:05.951846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd61083d-04d0-43f8-8ff7-a4a5067f1beb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:05.957930 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-fd61083d-04d0-43f8-8ff7-a4a5067f1beb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:05.957930 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:05.957930 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:05.957930 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:05.957930 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:05.957930 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:05.957930 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:05.957930 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:05.957930 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:05.957930 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:05.957930 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:05.957930 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:05.957930 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:05.957930 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:05.957930 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:05.957930 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:05.958942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:05.958942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:05.958942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:05.958942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:05.958942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:05.958942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:05.958942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:05.958942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:05.958942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:05.958942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:05.958942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:05.958942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:05.958942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:05.958942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:05.958942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:05.958942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:05.965245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:05.965245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:05.965245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:05.965245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:05.965245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:05.965245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:05.965245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:05.965245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:05.965245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:05.965245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:05.965245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:05.965245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:05.965245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 283/563] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Aug 5 19:50:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:05.965245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd61083d-04d0-43f8-8ff7-a4a5067f1beb None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:05.994526 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba1f97-59a1-4dfa-b7d9-7586666db81d None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:06.040950 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba6391c-5e71-4f31-bad0-42591f638b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:06.041607 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6391c-5e71-4f31-bad0-42591f638b95 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:06.041877 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6391c-5e71-4f31-bad0-42591f638b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:06.042147 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6391c-5e71-4f31-bad0-42591f638b95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:06.051134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba6391c-5e71-4f31-bad0-42591f638b95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:06.052628 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba6391c-5e71-4f31-bad0-42591f638b95 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:06.065186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-8ba6391c-5e71-4f31-bad0-42591f638b95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:06.065186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:06.065186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:06.065186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:06.065186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:06.065186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:06.065186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:06.065186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:06.065186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:06.065186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:06.065186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:06.065186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:06.065186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:06.065186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:06.065186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:06.066705 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:06.066705 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:06.066705 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:06.066705 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:06.066705 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:06.066705 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:06.066705 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:06.066705 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:06.066705 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:06.066705 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:06.066705 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:06.066705 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:06.066705 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:06.066705 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:06.066705 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:06.066705 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:06.066705 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:06.068255 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:06.068255 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:06.068255 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:06.068255 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:06.068255 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:06.068255 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:06.068255 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:06.068255 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:06.068255 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 281/564] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:06.068255 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e4958-38e4-4bc5-9e8e-a2323e8810d2 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:06.068255 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e4958-38e4-4bc5-9e8e-a2323e8810d2 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:06.069382 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e4958-38e4-4bc5-9e8e-a2323e8810d2 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:06.072778 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 282/565] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:06.074310 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e4958-38e4-4bc5-9e8e-a2323e8810d2 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:06.094462 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd61083d-04d0-43f8-8ff7-a4a5067f1beb None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:06.124680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:06.150466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392da24-8d35-43c0-9d72-2482b74b0480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:06.151466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392da24-8d35-43c0-9d72-2482b74b0480 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:06.151909 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392da24-8d35-43c0-9d72-2482b74b0480 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:06.152342 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392da24-8d35-43c0-9d72-2482b74b0480 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:06.156815 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:06.171680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392da24-8d35-43c0-9d72-2482b74b0480 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:06.173216 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392da24-8d35-43c0-9d72-2482b74b0480 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:06.186520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-b392da24-8d35-43c0-9d72-2482b74b0480 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:50:06.186520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:06.186520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:06.186520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:06.186520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:06.186520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:06.186520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:06.186520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:06.186520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:06.186520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:06.186520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:06.186520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:06.186520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:06.186520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:06.186520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:06.186520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:06.188240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:06.188240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:06.188240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:06.188240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:06.188240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:06.188240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:06.188240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:06.188240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:06.188240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:06.188240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:06.188240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:06.188240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:06.188240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:06.188240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:06.188240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:06.188240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:06.188240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:06.188240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:06.190001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:06.190001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:06.190001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:06.190001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:06.190001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:06.190001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:06.190001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:06.190001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:06.190001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:50:06.190001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:06.190001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 284/566] 10.222.0.39 () {56 vars in 1093 bytes} [Fri Aug 5 19:50:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:06.190001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392da24-8d35-43c0-9d72-2482b74b0480 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:06.248275 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e4958-38e4-4bc5-9e8e-a2323e8810d2 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:06.296144 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cefc20-7aaf-4fcf-8bbb-a9f5bd3e1a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:06.296948 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cefc20-7aaf-4fcf-8bbb-a9f5bd3e1a8b None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:06.297085 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cefc20-7aaf-4fcf-8bbb-a9f5bd3e1a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:06.297363 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cefc20-7aaf-4fcf-8bbb-a9f5bd3e1a8b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:06.308042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cefc20-7aaf-4fcf-8bbb-a9f5bd3e1a8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:06.313076 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-f0cefc20-7aaf-4fcf-8bbb-a9f5bd3e1a8b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:06.313076 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:06.313076 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:06.313076 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:06.313076 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:06.313076 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:06.313076 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:06.313076 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:06.313076 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:06.313076 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:06.313076 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:06.313076 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:06.313076 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:06.313076 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:06.313076 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:06.313076 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:06.314103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:06.314103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:06.314103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:06.314103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:06.314103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:06.314103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:06.314103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:06.314103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:06.314103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:06.314103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:06.314103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:06.314103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:06.314103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:06.314103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:06.314103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:06.314103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:06.314932 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:06.314932 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:06.314932 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:06.314932 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:06.314932 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:06.314932 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:06.314932 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:06.314932 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:06.314932 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:06.314932 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:06.314932 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:06.314932 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:06.314932 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 283/567] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Aug 5 19:50:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:06.315773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cefc20-7aaf-4fcf-8bbb-a9f5bd3e1a8b None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:06.339278 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392da24-8d35-43c0-9d72-2482b74b0480 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:06.385467 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e461021-dafe-4d57-9d6b-b1882da55296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:06.386168 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e461021-dafe-4d57-9d6b-b1882da55296 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:06.386450 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e461021-dafe-4d57-9d6b-b1882da55296 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:06.386724 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e461021-dafe-4d57-9d6b-b1882da55296 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:06.387443 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e461021-dafe-4d57-9d6b-b1882da55296 None admin] RBAC: Adding query filter params (name=service) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:06.388319 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e461021-dafe-4d57-9d6b-b1882da55296 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:06.389876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e461021-dafe-4d57-9d6b-b1882da55296 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:06.398619 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 285/568] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Aug 5 19:50:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:06.400203 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f9c7a-1f4c-49ac-a8c3-9a28a978340a None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:06.400490 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f9c7a-1f4c-49ac-a8c3-9a28a978340a None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:06.400805 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f9c7a-1f4c-49ac-a8c3-9a28a978340a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:06.421005 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-923f9c7a-1f4c-49ac-a8c3-9a28a978340a None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:06.422478 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-923f9c7a-1f4c-49ac-a8c3-9a28a978340a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:06.456732 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cefc20-7aaf-4fcf-8bbb-a9f5bd3e1a8b None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:06.460534 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:06.468134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:06.495769 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca1423-c658-4b73-b17b-e612949697de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:06.496435 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca1423-c658-4b73-b17b-e612949697de None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:06.496814 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca1423-c658-4b73-b17b-e612949697de None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:06.497219 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca1423-c658-4b73-b17b-e612949697de None admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:06.498143 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ca1423-c658-4b73-b17b-e612949697de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:06.498450 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ca1423-c658-4b73-b17b-e612949697de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:06.499883 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ca1423-c658-4b73-b17b-e612949697de None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:06.503576 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 286/569] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:50:06.504493 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f9c7a-1f4c-49ac-a8c3-9a28a978340a None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:06.521656 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 284/570] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:06.522571 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca1423-c658-4b73-b17b-e612949697de None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:06.639070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f9c7a-1f4c-49ac-a8c3-9a28a978340a None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:06.660816 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca1423-c658-4b73-b17b-e612949697de None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:06.698118 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7729f-651e-4c5b-9317-ce140cba12d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:06.698376 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25ea23-c1de-4e0a-b27e-c6804c6a21dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:06.698731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25ea23-c1de-4e0a-b27e-c6804c6a21dc None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:06.698833 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7729f-651e-4c5b-9317-ce140cba12d1 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:06.699009 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25ea23-c1de-4e0a-b27e-c6804c6a21dc None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:06.699111 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7729f-651e-4c5b-9317-ce140cba12d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:06.699276 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25ea23-c1de-4e0a-b27e-c6804c6a21dc None admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:06.699382 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7729f-651e-4c5b-9317-ce140cba12d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:06.700080 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25ea23-c1de-4e0a-b27e-c6804c6a21dc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:06.700220 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7729f-651e-4c5b-9317-ce140cba12d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:06.701089 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7729f-651e-4c5b-9317-ce140cba12d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:06.701652 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25ea23-c1de-4e0a-b27e-c6804c6a21dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:06.702348 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7729f-651e-4c5b-9317-ce140cba12d1 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:06.702611 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25ea23-c1de-4e0a-b27e-c6804c6a21dc None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:06.712534 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 285/571] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Aug 5 19:50:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:06.713933 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7729f-651e-4c5b-9317-ce140cba12d1 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:06.746093 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 287/572] 10.222.0.39 () {56 vars in 1133 bytes} [Fri Aug 5 19:50:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:06.747482 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25ea23-c1de-4e0a-b27e-c6804c6a21dc None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:06.769325 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:06.788072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:06.859675 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7729f-651e-4c5b-9317-ce140cba12d1 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:06.870056 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25ea23-c1de-4e0a-b27e-c6804c6a21dc None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:06.907965 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab7f2d-c8ec-46fd-9acf-3627a255f0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:06.908697 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab7f2d-c8ec-46fd-9acf-3627a255f0e9 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:06.908864 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20713311-369e-4c28-8d9f-15569c9253e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:06.909015 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab7f2d-c8ec-46fd-9acf-3627a255f0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:06.909146 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab7f2d-c8ec-46fd-9acf-3627a255f0e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:06.909505 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20713311-369e-4c28-8d9f-15569c9253e9 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:06.909783 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20713311-369e-4c28-8d9f-15569c9253e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:06.910053 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20713311-369e-4c28-8d9f-15569c9253e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:06.921971 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab7f2d-c8ec-46fd-9acf-3627a255f0e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:06.927474 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20713311-369e-4c28-8d9f-15569c9253e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:06.927655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-adab7f2d-c8ec-46fd-9acf-3627a255f0e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:06.927655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:06.927655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:06.927655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:06.927655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:06.927655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:06.927655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:06.927655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:06.927655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:06.927655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:06.927655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:06.927655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:06.927655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:06.927655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:06.927655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:06.927655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:06.929025 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:06.929025 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:06.929025 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:06.929025 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:06.929025 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:06.929025 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:06.929025 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:06.929025 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:06.929025 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:06.929025 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:06.929025 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:06.929025 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:06.929025 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:06.929025 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:06.929025 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:06.929025 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:06.930098 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:06.930098 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:06.930098 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:06.930098 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:06.930098 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:06.930098 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:06.930098 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:06.930098 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:06.930098 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:06.930098 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:06.930098 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:06.930098 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:06.930098 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 288/573] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Aug 5 19:50:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:06.930098 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab7f2d-c8ec-46fd-9acf-3627a255f0e9 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:06.932267 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20713311-369e-4c28-8d9f-15569c9253e9 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:06.948783 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-20713311-369e-4c28-8d9f-15569c9253e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:06.948783 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:06.948783 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:06.948783 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:06.948783 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:06.948783 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:06.948783 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:06.948783 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:06.948783 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:06.948783 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:06.948783 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:06.948783 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:06.948783 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:06.948783 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:06.948783 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:06.949831 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:06.949831 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:06.949831 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:06.949831 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:06.949831 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:06.949831 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:06.949831 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:06.949831 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:06.949831 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:06.949831 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:06.949831 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:06.949831 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:06.949831 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:06.949831 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:06.949831 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:06.949831 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:06.949831 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:06.950906 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:06.950906 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:06.950906 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:06.950906 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:06.950906 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 286/574] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:06.951423 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20713311-369e-4c28-8d9f-15569c9253e9 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:07.042487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:07.055215 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab7f2d-c8ec-46fd-9acf-3627a255f0e9 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:07.066047 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:07.087353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20713311-369e-4c28-8d9f-15569c9253e9 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:07.093544 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59f94c-789a-4ac9-9955-412c7acf9f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:07.094169 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59f94c-789a-4ac9-9955-412c7acf9f96 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:07.094452 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59f94c-789a-4ac9-9955-412c7acf9f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:07.094722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59f94c-789a-4ac9-9955-412c7acf9f96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:07.095521 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59f94c-789a-4ac9-9955-412c7acf9f96 None admin] RBAC: Adding query filter params (role.id=7ebfd9e58fe549f08cdaec5f906a38a6, scope.project.id=80379e2104644e98a2e85d4e4af1b365, user.id=91d3d43cb0bd4ca3b8e8f10e29cc421c) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:07.096317 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59f94c-789a-4ac9-9955-412c7acf9f96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:07.097834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59f94c-789a-4ac9-9955-412c7acf9f96 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:07.104344 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 289/575] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:50:06 2022] GET /identity/v3/role_assignments?user.id=91d3d43cb0bd4ca3b8e8f10e29cc421c&scope.project.id=80379e2104644e98a2e85d4e4af1b365&role.id=7ebfd9e58fe549f08cdaec5f906a38a6 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:07.105199 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59f94c-789a-4ac9-9955-412c7acf9f96 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:07.138916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6123864-be38-4246-9662-54801703b20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:07.140138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6123864-be38-4246-9662-54801703b20e None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:07.140417 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6123864-be38-4246-9662-54801703b20e None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:07.140734 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6123864-be38-4246-9662-54801703b20e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:07.161997 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6123864-be38-4246-9662-54801703b20e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:07.163915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6123864-be38-4246-9662-54801703b20e None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:07.184405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-c6123864-be38-4246-9662-54801703b20e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:07.184405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:07.184405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:07.184405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:07.184405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:07.184405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:07.184405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:07.184405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:07.184405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:07.184405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:07.184405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:07.184405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:07.184405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:07.184405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:07.184405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:07.185960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:07.185960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:07.185960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:07.185960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:07.185960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:07.185960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:07.185960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:07.185960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:07.185960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:07.185960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:07.185960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:07.185960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:07.185960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:07.185960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:07.185960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:07.185960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:07.185960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:07.186869 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:07.186869 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:07.186869 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:07.186869 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:07.186869 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 287/576] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:07.187538 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6123864-be38-4246-9662-54801703b20e None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:07.260744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59f94c-789a-4ac9-9955-412c7acf9f96 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:07.310240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a522c-013c-41a7-9c98-bd439e8562ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:07.311038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a522c-013c-41a7-9c98-bd439e8562ac None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:07.311369 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a522c-013c-41a7-9c98-bd439e8562ac None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:07.311707 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a522c-013c-41a7-9c98-bd439e8562ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:07.326841 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a522c-013c-41a7-9c98-bd439e8562ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:07.328660 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a522c-013c-41a7-9c98-bd439e8562ac None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:07.341556 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6123864-be38-4246-9662-54801703b20e None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:07.345924 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-895a522c-013c-41a7-9c98-bd439e8562ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:07.345924 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:07.345924 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:07.345924 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:07.345924 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:07.345924 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:07.345924 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:07.345924 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:07.345924 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:07.345924 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:07.345924 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:07.345924 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:07.345924 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:07.345924 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:07.345924 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:07.346819 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:07.346819 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:07.346819 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:07.346819 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:07.346819 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:07.346819 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:07.346819 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:07.346819 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:07.346819 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:07.346819 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:07.346819 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:07.346819 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:07.346819 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:07.346819 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:07.346819 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:07.346819 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:07.346819 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:07.353254 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:07.353254 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:07.353254 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:07.353254 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:07.353254 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 290/577] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:07.353254 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a522c-013c-41a7-9c98-bd439e8562ac None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:07.353254 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:07.406432 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6355778-3a4e-4a3f-b7ba-4f379dc287d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:07.407053 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6355778-3a4e-4a3f-b7ba-4f379dc287d3 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:07.407334 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6355778-3a4e-4a3f-b7ba-4f379dc287d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:07.407607 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6355778-3a4e-4a3f-b7ba-4f379dc287d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:07.408340 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6355778-3a4e-4a3f-b7ba-4f379dc287d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:07.408675 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6355778-3a4e-4a3f-b7ba-4f379dc287d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:07.410075 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6355778-3a4e-4a3f-b7ba-4f379dc287d3 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:07.411816 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:07.427976 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 288/578] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Aug 5 19:50:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:07.429578 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6355778-3a4e-4a3f-b7ba-4f379dc287d3 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:07.496970 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a522c-013c-41a7-9c98-bd439e8562ac None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:07.550982 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8fad1e-f67f-45c7-9316-e546642c6edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:07.551773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fad1e-f67f-45c7-9316-e546642c6edb None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:07.552110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fad1e-f67f-45c7-9316-e546642c6edb None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:07.552452 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fad1e-f67f-45c7-9316-e546642c6edb None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:07.553197 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8fad1e-f67f-45c7-9316-e546642c6edb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:07.553506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8fad1e-f67f-45c7-9316-e546642c6edb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:07.554908 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8fad1e-f67f-45c7-9316-e546642c6edb None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:07.573622 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 291/579] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:07.573622 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8fad1e-f67f-45c7-9316-e546642c6edb None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:07.578601 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6355778-3a4e-4a3f-b7ba-4f379dc287d3 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:07.619994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6bd4c6-f676-4499-87c1-d431b3a2a481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:07.620710 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6bd4c6-f676-4499-87c1-d431b3a2a481 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:07.621049 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6bd4c6-f676-4499-87c1-d431b3a2a481 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:07.621234 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6bd4c6-f676-4499-87c1-d431b3a2a481 None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:07.621960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6bd4c6-f676-4499-87c1-d431b3a2a481 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:07.622338 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6bd4c6-f676-4499-87c1-d431b3a2a481 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:07.624039 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6bd4c6-f676-4499-87c1-d431b3a2a481 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:07.643651 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 289/580] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:07.643651 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6bd4c6-f676-4499-87c1-d431b3a2a481 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:07.659849 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:07.718025 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8fad1e-f67f-45c7-9316-e546642c6edb None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:07.726366 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:07.772670 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da154e4d-6210-4c32-9e1d-a7f87d077224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:07.772963 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da154e4d-6210-4c32-9e1d-a7f87d077224 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:07.773194 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da154e4d-6210-4c32-9e1d-a7f87d077224 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:07.773683 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da154e4d-6210-4c32-9e1d-a7f87d077224 None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:07.774465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da154e4d-6210-4c32-9e1d-a7f87d077224 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:07.774764 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da154e4d-6210-4c32-9e1d-a7f87d077224 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:07.776287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da154e4d-6210-4c32-9e1d-a7f87d077224 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:07.789735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 292/581] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:07.790356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da154e4d-6210-4c32-9e1d-a7f87d077224 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:07.805357 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6bd4c6-f676-4499-87c1-d431b3a2a481 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:07.850015 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127264c9-0ea5-4900-8bc8-6536ddef92f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:07.850630 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127264c9-0ea5-4900-8bc8-6536ddef92f4 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:07.850915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127264c9-0ea5-4900-8bc8-6536ddef92f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:07.851212 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127264c9-0ea5-4900-8bc8-6536ddef92f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:07.864233 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127264c9-0ea5-4900-8bc8-6536ddef92f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:07.865993 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127264c9-0ea5-4900-8bc8-6536ddef92f4 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:07.882118 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-127264c9-0ea5-4900-8bc8-6536ddef92f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:07.882118 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:07.882118 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:07.882118 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:07.882118 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:07.882118 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:07.882118 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:07.882118 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:07.882118 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:07.882118 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:07.882118 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:07.882118 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:07.882118 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:07.882118 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:07.882118 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:07.883247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:07.883247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:07.883247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:07.883247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:07.883247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:07.883247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:07.883247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:07.883247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:07.883247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:07.883247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:07.883247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:07.883247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:07.883247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:07.883247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:07.883247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:07.883247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:07.883247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:07.884130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:07.884130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:07.884130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:07.884130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:07.884130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 290/582] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:07.884130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127264c9-0ea5-4900-8bc8-6536ddef92f4 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:07.917223 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da154e4d-6210-4c32-9e1d-a7f87d077224 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:07.955040 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:07.968230 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ab872-e766-4660-9fcb-e064fe59527d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:07.968904 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ab872-e766-4660-9fcb-e064fe59527d None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:07.969190 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ab872-e766-4660-9fcb-e064fe59527d None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:07.969464 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ab872-e766-4660-9fcb-e064fe59527d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:07.980105 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ab872-e766-4660-9fcb-e064fe59527d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:07.982110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ab872-e766-4660-9fcb-e064fe59527d None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:07.995016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-542ab872-e766-4660-9fcb-e064fe59527d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 19:50:07.995016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:07.995016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:07.995016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:07.995016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:07.995016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:07.995016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:07.995016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:07.995016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:07.995016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:07.995016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:07.995016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:07.995016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:07.995016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:07.995016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:07.995016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:07.996007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:07.996007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:07.996007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:07.996007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:07.996007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:07.996007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:07.996007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:07.996007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:07.996007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:07.996007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:07.996007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:07.996007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:07.996007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:07.996007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:07.996007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:07.996007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:07.996007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:07.996007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:07.997286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:07.997286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:07.997286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:07.997286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:07.997286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:07.997286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:07.997286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:07.997286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:07.997286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 19:50:07.997286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:07.997286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 293/583] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:50:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:07.997286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ab872-e766-4660-9fcb-e064fe59527d None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:08.015685 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127264c9-0ea5-4900-8bc8-6536ddef92f4 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:08.029272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:08.067802 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653159c-cb79-4aeb-94d0-5cd56bcc3dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:08.068447 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653159c-cb79-4aeb-94d0-5cd56bcc3dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:08.068764 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653159c-cb79-4aeb-94d0-5cd56bcc3dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:08.069109 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653159c-cb79-4aeb-94d0-5cd56bcc3dd3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:08.078919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3653159c-cb79-4aeb-94d0-5cd56bcc3dd3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:08.080430 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3653159c-cb79-4aeb-94d0-5cd56bcc3dd3 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:08.093244 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-3653159c-cb79-4aeb-94d0-5cd56bcc3dd3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 05 19:50:08.093244 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:08.093244 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:08.093244 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:08.093244 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:08.093244 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:08.093244 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:08.093244 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:08.093244 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:08.093244 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:08.093244 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:08.093244 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:08.093244 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:08.093244 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:08.093244 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:08.093244 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:08.094257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:08.094257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:08.094257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:08.094257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:08.094257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:08.094257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:08.094257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:08.094257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:08.094257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:08.094257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:08.094257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:08.094257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:08.094257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:08.094257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:08.094257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:08.094257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:08.094257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:08.094257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:08.095186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:08.095186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:08.095186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:08.095186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:08.095186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:08.095186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:08.095186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:08.095186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:08.095186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 05 19:50:08.095186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:08.095186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 291/584] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:50:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:08.096464 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653159c-cb79-4aeb-94d0-5cd56bcc3dd3 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:08.142175 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ab872-e766-4660-9fcb-e064fe59527d None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:08.193608 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad11138-d7e3-4b5b-9e5f-3237f954db6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:08.194368 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad11138-d7e3-4b5b-9e5f-3237f954db6b None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:08.194647 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad11138-d7e3-4b5b-9e5f-3237f954db6b None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:08.194920 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad11138-d7e3-4b5b-9e5f-3237f954db6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:08.202869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad11138-d7e3-4b5b-9e5f-3237f954db6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:08.204341 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad11138-d7e3-4b5b-9e5f-3237f954db6b None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:08.222621 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-bad11138-d7e3-4b5b-9e5f-3237f954db6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:08.222621 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:08.222621 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:08.222621 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:08.222621 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:08.222621 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:08.222621 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:08.222621 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:08.222621 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:08.222621 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:08.222621 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:08.222621 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:08.222621 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:08.222621 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:08.222621 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:08.223584 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:08.223584 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:08.223584 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:08.223584 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:08.223584 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:08.223584 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:08.223584 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:08.223584 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:08.223584 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:08.223584 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:08.223584 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:08.223584 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:08.223584 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:08.223584 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:08.223584 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:08.223584 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:08.223584 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:08.224469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:08.224469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:08.224469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:08.224469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:08.224469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:08.224469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:08.224469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:08.224469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:08.225403 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 294/585] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:08.226458 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad11138-d7e3-4b5b-9e5f-3237f954db6b None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:08.242786 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653159c-cb79-4aeb-94d0-5cd56bcc3dd3 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:08.281228 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2979881-647d-4548-8108-d7fa3ec574a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:08.281975 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2979881-647d-4548-8108-d7fa3ec574a9 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:08.282236 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2979881-647d-4548-8108-d7fa3ec574a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:08.282397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2979881-647d-4548-8108-d7fa3ec574a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:08.283094 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2979881-647d-4548-8108-d7fa3ec574a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:08.283401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2979881-647d-4548-8108-d7fa3ec574a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:08.284900 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2979881-647d-4548-8108-d7fa3ec574a9 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:08.298519 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 292/586] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:08.299390 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2979881-647d-4548-8108-d7fa3ec574a9 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:08.315206 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:08.339242 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:08.395394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad11138-d7e3-4b5b-9e5f-3237f954db6b None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:08.438288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2979881-647d-4548-8108-d7fa3ec574a9 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:08.444470 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a69694e-9fd7-4527-9ab7-682d73545db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:08.445248 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69694e-9fd7-4527-9ab7-682d73545db4 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:08.445676 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69694e-9fd7-4527-9ab7-682d73545db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:08.446036 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69694e-9fd7-4527-9ab7-682d73545db4 None admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:08.447628 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a69694e-9fd7-4527-9ab7-682d73545db4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:08.448031 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a69694e-9fd7-4527-9ab7-682d73545db4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:08.451492 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a69694e-9fd7-4527-9ab7-682d73545db4 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:08.473704 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 295/587] 10.222.0.39 () {56 vars in 1137 bytes} [Fri Aug 5 19:50:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:08.474761 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a69694e-9fd7-4527-9ab7-682d73545db4 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:08.487133 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b5bb8-e009-49bc-abbb-121f00627570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:08.488188 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b5bb8-e009-49bc-abbb-121f00627570 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:08.488745 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b5bb8-e009-49bc-abbb-121f00627570 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:08.489099 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b5bb8-e009-49bc-abbb-121f00627570 None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:08.490230 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096b5bb8-e009-49bc-abbb-121f00627570 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:08.490766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096b5bb8-e009-49bc-abbb-121f00627570 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:08.493293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096b5bb8-e009-49bc-abbb-121f00627570 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:08.528949 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 293/588] 10.222.0.39 () {56 vars in 1137 bytes} [Fri Aug 5 19:50:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:08.530254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b5bb8-e009-49bc-abbb-121f00627570 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:08.655949 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a69694e-9fd7-4527-9ab7-682d73545db4 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:08.657900 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:08.670520 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:08.691232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b5bb8-e009-49bc-abbb-121f00627570 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:08.721976 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71d095-e06a-4240-902c-a3121749cf4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:08.722610 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71d095-e06a-4240-902c-a3121749cf4e None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:08.722890 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71d095-e06a-4240-902c-a3121749cf4e None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:08.723162 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71d095-e06a-4240-902c-a3121749cf4e None admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:08.723919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e71d095-e06a-4240-902c-a3121749cf4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:08.724224 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e71d095-e06a-4240-902c-a3121749cf4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:08.725719 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e71d095-e06a-4240-902c-a3121749cf4e None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:08.743245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 296/589] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:08.744541 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71d095-e06a-4240-902c-a3121749cf4e None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:08.746020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6c7ca-fd41-4f77-81c8-eedb838ce8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:08.747978 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6c7ca-fd41-4f77-81c8-eedb838ce8a2 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:08.748324 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6c7ca-fd41-4f77-81c8-eedb838ce8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:08.748893 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6c7ca-fd41-4f77-81c8-eedb838ce8a2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:08.757496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e6c7ca-fd41-4f77-81c8-eedb838ce8a2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:08.759119 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e6c7ca-fd41-4f77-81c8-eedb838ce8a2 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:08.779078 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-01e6c7ca-fd41-4f77-81c8-eedb838ce8a2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 19:50:08.779078 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:08.779078 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:08.779078 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:08.779078 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:08.779078 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:08.779078 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:08.779078 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:08.779078 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:08.779078 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:08.779078 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:08.779078 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:08.779078 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:08.779078 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:08.779078 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:08.779078 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:08.780935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:08.780935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:08.780935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:08.780935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:08.780935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:08.780935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:08.780935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:08.780935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:08.780935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:08.780935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:08.780935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:08.780935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:08.780935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:08.780935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:08.780935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:08.780935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:08.780935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:08.780935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:08.787175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:08.787175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:08.787175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:08.787175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:08.787175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:08.787175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:08.787175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:08.787175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:08.787175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 19:50:08.787175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:08.787175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 294/590] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Aug 5 19:50:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:08.787175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6c7ca-fd41-4f77-81c8-eedb838ce8a2 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:08.910231 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71d095-e06a-4240-902c-a3121749cf4e None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:08.953462 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6c7ca-fd41-4f77-81c8-eedb838ce8a2 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:08.968400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72335978-4fb1-4c1d-bd54-a08d276b2fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:08.969387 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72335978-4fb1-4c1d-bd54-a08d276b2fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:08.969670 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72335978-4fb1-4c1d-bd54-a08d276b2fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:08.969943 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72335978-4fb1-4c1d-bd54-a08d276b2fb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:08.987527 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72335978-4fb1-4c1d-bd54-a08d276b2fb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:08.989023 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72335978-4fb1-4c1d-bd54-a08d276b2fb9 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:09.007772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-72335978-4fb1-4c1d-bd54-a08d276b2fb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:09.007772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:09.007772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:09.007772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:09.007772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:09.007772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:09.007772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:09.007772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:09.007772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:09.007772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:09.007772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:09.007772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:09.007772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:09.007772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:09.007772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:09.009579 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:09.009579 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:09.009579 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:09.009579 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:09.009579 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:09.009579 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:09.009579 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:09.009579 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:09.009579 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:09.009579 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:09.009579 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:09.009579 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:09.009579 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:09.009579 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:09.009579 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:09.009579 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:09.009579 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:09.011503 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:09.011503 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:09.011503 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:09.011503 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:09.013212 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 297/591] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:09.014551 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587de0b5-a5e9-4b36-8586-9a40304fefc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:09.014895 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72335978-4fb1-4c1d-bd54-a08d276b2fb9 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:09.015215 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587de0b5-a5e9-4b36-8586-9a40304fefc5 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:09.015612 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587de0b5-a5e9-4b36-8586-9a40304fefc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:09.015894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587de0b5-a5e9-4b36-8586-9a40304fefc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:09.030434 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587de0b5-a5e9-4b36-8586-9a40304fefc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:09.031898 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587de0b5-a5e9-4b36-8586-9a40304fefc5 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:09.035556 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:09.040791 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:09.051167 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-587de0b5-a5e9-4b36-8586-9a40304fefc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:09.051167 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:09.051167 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:09.051167 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:09.051167 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:09.051167 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:09.051167 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:09.051167 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:09.051167 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:09.051167 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:09.051167 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:09.051167 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:09.051167 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:09.051167 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:09.051167 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:09.052170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:09.052170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:09.052170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:09.052170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:09.052170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:09.052170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:09.052170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:09.052170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:09.052170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:09.052170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:09.052170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:09.052170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:09.052170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:09.052170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:09.052170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:09.052170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:09.052170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:09.053135 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:09.053135 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:09.053135 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:09.053135 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:09.053135 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 295/592] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:09.054285 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587de0b5-a5e9-4b36-8586-9a40304fefc5 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:09.158158 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72335978-4fb1-4c1d-bd54-a08d276b2fb9 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:09.192367 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587de0b5-a5e9-4b36-8586-9a40304fefc5 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:09.213002 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120ab9d-12ca-4dea-95af-af6182469bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:09.213680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120ab9d-12ca-4dea-95af-af6182469bbb None admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:09.214047 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120ab9d-12ca-4dea-95af-af6182469bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:09.214327 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120ab9d-12ca-4dea-95af-af6182469bbb None admin] PATH_INFO: `/v3/projects/80379e2104644e98a2e85d4e4af1b365/users/95e9f085e4f04d118373b8abff27543a/roles/7ebfd9e58fe549f08cdaec5f906a38a6` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:09.236003 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a120ab9d-12ca-4dea-95af-af6182469bbb None admin] RBAC: Authorizing `identity:create_grant(project_id=80379e2104644e98a2e85d4e4af1b365, user_id=95e9f085e4f04d118373b8abff27543a, role_id=7ebfd9e58fe549f08cdaec5f906a38a6)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:09.238150 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a120ab9d-12ca-4dea-95af-af6182469bbb None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:09.241848 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e682e1-ea6c-4c38-a597-88cc2af988c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:09.242577 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e682e1-ea6c-4c38-a597-88cc2af988c8 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:09.242963 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e682e1-ea6c-4c38-a597-88cc2af988c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:09.243265 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e682e1-ea6c-4c38-a597-88cc2af988c8 None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:09.244045 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e682e1-ea6c-4c38-a597-88cc2af988c8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:09.244349 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e682e1-ea6c-4c38-a597-88cc2af988c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:09.246173 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e682e1-ea6c-4c38-a597-88cc2af988c8 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:09.254136 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 298/593] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Aug 5 19:50:09 2022] PUT /identity/v3/projects/80379e2104644e98a2e85d4e4af1b365/users/95e9f085e4f04d118373b8abff27543a/roles/7ebfd9e58fe549f08cdaec5f906a38a6 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:09.254664 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120ab9d-12ca-4dea-95af-af6182469bbb None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:09.294730 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 296/594] 10.222.0.39 () {56 vars in 1139 bytes} [Fri Aug 5 19:50:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:09.295230 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e682e1-ea6c-4c38-a597-88cc2af988c8 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:09.345123 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:09.368078 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:09.404018 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120ab9d-12ca-4dea-95af-af6182469bbb None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:09.425440 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e682e1-ea6c-4c38-a597-88cc2af988c8 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:09.447042 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8679d5-36a2-45f9-84a3-ba2da215a902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:09.447657 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8679d5-36a2-45f9-84a3-ba2da215a902 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:09.447980 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8679d5-36a2-45f9-84a3-ba2da215a902 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:09.448203 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8679d5-36a2-45f9-84a3-ba2da215a902 None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:09.450663 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8679d5-36a2-45f9-84a3-ba2da215a902 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:09.450663 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8679d5-36a2-45f9-84a3-ba2da215a902 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:09.451311 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8679d5-36a2-45f9-84a3-ba2da215a902 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:09.463936 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 299/595] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:09.465240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577da7dc-5a26-487a-8a95-9d57acf60b7f None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:09.465534 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577da7dc-5a26-487a-8a95-9d57acf60b7f None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:09.466361 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577da7dc-5a26-487a-8a95-9d57acf60b7f None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:09.471480 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 300/596] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:09.471480 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577da7dc-5a26-487a-8a95-9d57acf60b7f None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:09.473420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5cbb43-0243-48c3-8c98-01a2c73fe861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:09.474044 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cbb43-0243-48c3-8c98-01a2c73fe861 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:09.474320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cbb43-0243-48c3-8c98-01a2c73fe861 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:09.474591 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cbb43-0243-48c3-8c98-01a2c73fe861 None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:09.475280 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5cbb43-0243-48c3-8c98-01a2c73fe861 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:09.475581 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5cbb43-0243-48c3-8c98-01a2c73fe861 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:09.476989 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5cbb43-0243-48c3-8c98-01a2c73fe861 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:09.489996 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 297/597] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:09.490677 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5cbb43-0243-48c3-8c98-01a2c73fe861 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:09.591845 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577da7dc-5a26-487a-8a95-9d57acf60b7f None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:09.613228 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5cbb43-0243-48c3-8c98-01a2c73fe861 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:09.627201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:09.638734 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:09.643012 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c02af7-7934-4d53-9b1c-415abc39c283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:09.643658 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c02af7-7934-4d53-9b1c-415abc39c283 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:09.643944 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c02af7-7934-4d53-9b1c-415abc39c283 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:09.644222 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c02af7-7934-4d53-9b1c-415abc39c283 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:09.659009 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c02af7-7934-4d53-9b1c-415abc39c283 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:09.660600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c02af7-7934-4d53-9b1c-415abc39c283 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:09.661037 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724002c9-b15c-42f1-a3ba-26003695317b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:09.661839 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724002c9-b15c-42f1-a3ba-26003695317b None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:09.662206 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724002c9-b15c-42f1-a3ba-26003695317b None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:09.662591 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724002c9-b15c-42f1-a3ba-26003695317b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:09.671006 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724002c9-b15c-42f1-a3ba-26003695317b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:09.672619 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724002c9-b15c-42f1-a3ba-26003695317b None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:09.678634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-67c02af7-7934-4d53-9b1c-415abc39c283 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:09.678634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:09.678634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:09.678634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:09.678634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:09.678634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:09.678634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:09.678634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:09.678634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:09.678634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:09.678634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:09.678634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:09.678634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:09.678634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:09.678634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:09.679506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:09.679506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:09.679506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:09.679506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:09.679506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:09.679506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:09.679506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:09.679506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:09.679506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:09.679506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:09.679506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:09.679506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:09.679506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:09.679506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:09.679506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:09.679506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:09.679506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:09.681362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:09.681362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:09.681362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:09.681362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:09.681362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 301/598] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:09.681362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5c4a7-cd76-4696-8f3d-527616b31263 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:09.681972 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5c4a7-cd76-4696-8f3d-527616b31263 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:09.681972 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5c4a7-cd76-4696-8f3d-527616b31263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:09.691262 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-724002c9-b15c-42f1-a3ba-26003695317b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:09.691262 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:09.691262 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:09.691262 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:09.691262 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:09.691262 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:09.691262 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:09.691262 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:09.691262 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:09.691262 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:09.691262 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:09.691262 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:09.691262 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:09.691262 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:09.691262 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:09.694887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:09.694887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:09.694887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:09.694887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:09.694887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:09.694887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:09.694887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:09.694887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:09.694887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:09.694887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:09.694887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:09.694887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:09.694887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:09.694887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:09.694887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:09.694887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:09.694887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:09.696887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:09.696887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:09.696887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:09.696887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:09.696887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:09.696887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:09.696887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:09.696887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:09.696887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 298/599] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:09.696887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724002c9-b15c-42f1-a3ba-26003695317b None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:09.710083 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-03b5c4a7-cd76-4696-8f3d-527616b31263 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:09.711172 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-03b5c4a7-cd76-4696-8f3d-527616b31263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:09.798809 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 302/600] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:50:09.800330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b5c4a7-cd76-4696-8f3d-527616b31263 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:09.827319 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724002c9-b15c-42f1-a3ba-26003695317b None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:09.865715 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784524e4-ff6d-48ce-92d5-086fbaa79796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:09.866526 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784524e4-ff6d-48ce-92d5-086fbaa79796 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:09.866868 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784524e4-ff6d-48ce-92d5-086fbaa79796 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:09.867240 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784524e4-ff6d-48ce-92d5-086fbaa79796 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:09.877526 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784524e4-ff6d-48ce-92d5-086fbaa79796 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:09.879113 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784524e4-ff6d-48ce-92d5-086fbaa79796 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:09.892253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-784524e4-ff6d-48ce-92d5-086fbaa79796 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:09.892253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:09.892253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:09.892253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:09.892253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:09.892253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:09.892253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:09.892253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:09.892253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:09.892253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:09.892253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:09.892253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:09.892253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:09.892253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:09.892253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:09.893409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:09.893409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:09.893409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:09.893409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:09.893409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:09.893409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:09.893409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:09.893409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:09.893409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:09.893409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:09.893409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:09.893409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:09.893409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:09.893409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:09.893409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:09.893409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:09.893409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:09.894429 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:09.894429 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:09.894429 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:09.894429 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:09.894429 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:09.894429 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:09.894429 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:09.894429 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:09.894429 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 299/601] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:09.894918 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784524e4-ff6d-48ce-92d5-086fbaa79796 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:09.938074 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b5c4a7-cd76-4696-8f3d-527616b31263 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:09.939083 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:09.949338 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:09.991722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625fc786-2d8c-4932-abdb-2ffa25ecebe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:09.992363 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625fc786-2d8c-4932-abdb-2ffa25ecebe0 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:09.992637 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625fc786-2d8c-4932-abdb-2ffa25ecebe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:09.993012 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625fc786-2d8c-4932-abdb-2ffa25ecebe0 None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:09.993863 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625fc786-2d8c-4932-abdb-2ffa25ecebe0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:09.994170 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625fc786-2d8c-4932-abdb-2ffa25ecebe0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:09.995557 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625fc786-2d8c-4932-abdb-2ffa25ecebe0 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:10.025152 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 303/602] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:10.025152 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625fc786-2d8c-4932-abdb-2ffa25ecebe0 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:10.046718 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784524e4-ff6d-48ce-92d5-086fbaa79796 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:10.094870 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6900fcbc-3343-440c-9118-f4ac5be96299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:10.095614 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900fcbc-3343-440c-9118-f4ac5be96299 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:10.095968 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900fcbc-3343-440c-9118-f4ac5be96299 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:10.096375 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900fcbc-3343-440c-9118-f4ac5be96299 None admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:10.097775 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6900fcbc-3343-440c-9118-f4ac5be96299 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:10.098301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6900fcbc-3343-440c-9118-f4ac5be96299 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:10.100170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6900fcbc-3343-440c-9118-f4ac5be96299 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:10.120784 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 300/603] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:10.122353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6900fcbc-3343-440c-9118-f4ac5be96299 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:10.173145 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625fc786-2d8c-4932-abdb-2ffa25ecebe0 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:10.237993 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b155b9-ec3e-4ebd-96d2-fcaeca2b488e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:10.238917 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b155b9-ec3e-4ebd-96d2-fcaeca2b488e None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:10.239381 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b155b9-ec3e-4ebd-96d2-fcaeca2b488e None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:10.239815 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b155b9-ec3e-4ebd-96d2-fcaeca2b488e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:10.252281 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b155b9-ec3e-4ebd-96d2-fcaeca2b488e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:10.258341 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-07b155b9-ec3e-4ebd-96d2-fcaeca2b488e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:10.258341 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:10.258341 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:10.258341 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:10.258341 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:10.258341 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:10.258341 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:10.258341 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:10.258341 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:10.258341 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:10.258341 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:10.258341 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:10.258341 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:10.258341 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:10.258341 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:10.258341 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:10.260008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:10.260008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:10.260008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:10.260008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:10.260008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:10.260008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:10.260008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:10.260008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:10.260008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:10.260008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:10.260008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:10.260008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:10.260008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:10.260008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:10.260008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:10.260008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:10.261580 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:10.261580 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:10.261580 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:10.261580 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:10.261580 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:10.261580 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:10.261580 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:10.261580 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:10.261580 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:10.261580 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:10.261580 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:10.261580 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:10.265089 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 304/604] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Aug 5 19:50:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:10.267283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b155b9-ec3e-4ebd-96d2-fcaeca2b488e None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:10.274644 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:10.285132 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6900fcbc-3343-440c-9118-f4ac5be96299 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:10.310191 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:10.336856 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e6209-7229-4398-955e-6eff4c42d7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:10.337765 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e6209-7229-4398-955e-6eff4c42d7a4 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:10.338532 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e6209-7229-4398-955e-6eff4c42d7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:10.338911 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e6209-7229-4398-955e-6eff4c42d7a4 None admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:10.340013 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e6209-7229-4398-955e-6eff4c42d7a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:10.340368 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e6209-7229-4398-955e-6eff4c42d7a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:10.342485 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e6209-7229-4398-955e-6eff4c42d7a4 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:10.355993 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 301/605] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:10.358399 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e6209-7229-4398-955e-6eff4c42d7a4 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:10.431314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b155b9-ec3e-4ebd-96d2-fcaeca2b488e None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:10.480110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b88c0-a4b7-4798-82a4-b40d11915745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:10.480967 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b88c0-a4b7-4798-82a4-b40d11915745 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:10.481399 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b88c0-a4b7-4798-82a4-b40d11915745 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:10.481947 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b88c0-a4b7-4798-82a4-b40d11915745 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:10.490035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1b88c0-a4b7-4798-82a4-b40d11915745 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:10.491585 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1b88c0-a4b7-4798-82a4-b40d11915745 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:10.514303 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e6209-7229-4398-955e-6eff4c42d7a4 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:10.517010 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-3c1b88c0-a4b7-4798-82a4-b40d11915745 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:10.517010 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:10.517010 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:10.517010 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:10.517010 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:10.517010 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:10.517010 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:10.517010 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:10.517010 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:10.517010 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:10.517010 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:10.517010 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:10.517010 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:10.517010 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:10.517010 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:10.519213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:10.519213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:10.519213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:10.519213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:10.519213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:10.519213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:10.519213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:10.519213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:10.519213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:10.519213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:10.519213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:10.519213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:10.519213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:10.519213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:10.519213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:10.519213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:10.519213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:10.521792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:10.521792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:10.521792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:10.521792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:10.521792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:10.521792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:10.521792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:10.521792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:10.521792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 305/606] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:10.521792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b88c0-a4b7-4798-82a4-b40d11915745 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:10.558889 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88748e-abc9-420e-b5ae-7a4eccede059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:10.560610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88748e-abc9-420e-b5ae-7a4eccede059 None admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:10.560987 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88748e-abc9-420e-b5ae-7a4eccede059 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:10.561269 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88748e-abc9-420e-b5ae-7a4eccede059 None admin] PATH_INFO: `/v3/projects/80379e2104644e98a2e85d4e4af1b365/users/44dfe45766d743558ed82e464721cf72/roles/7ebfd9e58fe549f08cdaec5f906a38a6` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:10.577598 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c88748e-abc9-420e-b5ae-7a4eccede059 None admin] RBAC: Authorizing `identity:create_grant(project_id=80379e2104644e98a2e85d4e4af1b365, user_id=44dfe45766d743558ed82e464721cf72, role_id=7ebfd9e58fe549f08cdaec5f906a38a6)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:10.579317 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c88748e-abc9-420e-b5ae-7a4eccede059 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:10.594308 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 302/607] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Aug 5 19:50:10 2022] PUT /identity/v3/projects/80379e2104644e98a2e85d4e4af1b365/users/44dfe45766d743558ed82e464721cf72/roles/7ebfd9e58fe549f08cdaec5f906a38a6 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:10.605118 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88748e-abc9-420e-b5ae-7a4eccede059 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:10.618602 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:10.709909 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:10.742043 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b88c0-a4b7-4798-82a4-b40d11915745 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:10.769228 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88748e-abc9-420e-b5ae-7a4eccede059 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:10.810597 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ba373-b281-4836-b522-f98de129ab50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:10.811969 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ba373-b281-4836-b522-f98de129ab50 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:10.812759 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ba373-b281-4836-b522-f98de129ab50 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:10.813437 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ba373-b281-4836-b522-f98de129ab50 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:10.814965 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ba373-b281-4836-b522-f98de129ab50 None admin] RBAC: Adding query filter params (name=service) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:10.815586 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f79c10-0d67-4023-9c56-0380e70ad69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:10.815991 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ba373-b281-4836-b522-f98de129ab50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:10.816406 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f79c10-0d67-4023-9c56-0380e70ad69d None admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:10.816778 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f79c10-0d67-4023-9c56-0380e70ad69d None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:10.817122 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f79c10-0d67-4023-9c56-0380e70ad69d None admin] PATH_INFO: `/v3/projects/80379e2104644e98a2e85d4e4af1b365/users/2fe69b398c1f4e18b625a63ae40b01e8/roles/7ebfd9e58fe549f08cdaec5f906a38a6` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:10.819689 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ba373-b281-4836-b522-f98de129ab50 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:10.834075 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f79c10-0d67-4023-9c56-0380e70ad69d None admin] RBAC: Authorizing `identity:create_grant(project_id=80379e2104644e98a2e85d4e4af1b365, user_id=2fe69b398c1f4e18b625a63ae40b01e8, role_id=7ebfd9e58fe549f08cdaec5f906a38a6)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:10.835643 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f79c10-0d67-4023-9c56-0380e70ad69d None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:10.836759 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 306/608] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Aug 5 19:50:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:10.838603 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ba373-b281-4836-b522-f98de129ab50 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:10.865059 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 303/609] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Aug 5 19:50:10 2022] PUT /identity/v3/projects/80379e2104644e98a2e85d4e4af1b365/users/2fe69b398c1f4e18b625a63ae40b01e8/roles/7ebfd9e58fe549f08cdaec5f906a38a6 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:10.866879 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f79c10-0d67-4023-9c56-0380e70ad69d None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:11.053073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ba373-b281-4836-b522-f98de129ab50 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:11.077897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:11.098889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb2bbf2-63b0-4710-9fc9-929978854aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:11.100105 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2bbf2-63b0-4710-9fc9-929978854aab None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:11.100607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2bbf2-63b0-4710-9fc9-929978854aab None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:11.101121 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2bbf2-63b0-4710-9fc9-929978854aab None admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:11.102335 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb2bbf2-63b0-4710-9fc9-929978854aab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:11.102818 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb2bbf2-63b0-4710-9fc9-929978854aab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:11.104547 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb2bbf2-63b0-4710-9fc9-929978854aab None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:11.132548 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 307/610] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:11.134203 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb2bbf2-63b0-4710-9fc9-929978854aab None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:11.150305 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:11.213211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f79c10-0d67-4023-9c56-0380e70ad69d None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:11.300912 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb2bbf2-63b0-4710-9fc9-929978854aab None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:11.311237 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d31c97-a02f-4f13-988e-79bac3cbdf04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:11.311812 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d31c97-a02f-4f13-988e-79bac3cbdf04 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:11.312086 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d31c97-a02f-4f13-988e-79bac3cbdf04 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:11.312356 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d31c97-a02f-4f13-988e-79bac3cbdf04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:11.330817 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d31c97-a02f-4f13-988e-79bac3cbdf04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:11.332275 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d31c97-a02f-4f13-988e-79bac3cbdf04 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:11.362042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-14d31c97-a02f-4f13-988e-79bac3cbdf04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:11.362042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:11.362042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:11.362042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:11.362042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:11.362042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:11.362042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:11.362042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:11.362042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:11.362042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:11.362042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:11.362042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:11.362042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:11.362042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:11.362042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:11.362997 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:11.362997 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:11.362997 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:11.362997 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:11.362997 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:11.362997 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:11.362997 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:11.362997 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:11.362997 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:11.362997 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:11.362997 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:11.362997 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:11.362997 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:11.362997 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:11.362997 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:11.362997 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:11.362997 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:11.363887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:11.363887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:11.363887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:11.363887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:11.365013 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 304/611] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 499 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:11.372685 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d31c97-a02f-4f13-988e-79bac3cbdf04 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:11.377546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30da5e-3149-46cc-a7b1-83d5478b96f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:11.378536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30da5e-3149-46cc-a7b1-83d5478b96f3 None admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:11.379688 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30da5e-3149-46cc-a7b1-83d5478b96f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:11.382121 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30da5e-3149-46cc-a7b1-83d5478b96f3 None admin] PATH_INFO: `/v3/projects/80379e2104644e98a2e85d4e4af1b365/users/b1c75368c00e4b0cb7b3163728843479/roles/7ebfd9e58fe549f08cdaec5f906a38a6` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:11.409571 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed30da5e-3149-46cc-a7b1-83d5478b96f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=80379e2104644e98a2e85d4e4af1b365, user_id=b1c75368c00e4b0cb7b3163728843479, role_id=7ebfd9e58fe549f08cdaec5f906a38a6)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:11.413285 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed30da5e-3149-46cc-a7b1-83d5478b96f3 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:11.440753 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 308/612] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Aug 5 19:50:11 2022] PUT /identity/v3/projects/80379e2104644e98a2e85d4e4af1b365/users/b1c75368c00e4b0cb7b3163728843479/roles/7ebfd9e58fe549f08cdaec5f906a38a6 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:50:11.515217 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc8a48-c09a-49cb-925a-a63cd7e741fc None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:11.515645 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc8a48-c09a-49cb-925a-a63cd7e741fc None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:11.515933 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc8a48-c09a-49cb-925a-a63cd7e741fc None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:11.519556 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 309/613] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:50:11.530245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b2cc8-dd5b-4c85-b997-340fb20bb5b0 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:11.530572 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b2cc8-dd5b-4c85-b997-340fb20bb5b0 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:11.530808 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b2cc8-dd5b-4c85-b997-340fb20bb5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:11.553243 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-038b2cc8-dd5b-4c85-b997-340fb20bb5b0 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:11.554637 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-038b2cc8-dd5b-4c85-b997-340fb20bb5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:11.669626 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d31c97-a02f-4f13-988e-79bac3cbdf04 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:11.688674 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:11.702134 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:11.732245 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e47a703-71f9-4811-9b2a-da64c94d6100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:11.733300 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47a703-71f9-4811-9b2a-da64c94d6100 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:11.733651 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47a703-71f9-4811-9b2a-da64c94d6100 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:11.733994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47a703-71f9-4811-9b2a-da64c94d6100 None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:11.734778 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e47a703-71f9-4811-9b2a-da64c94d6100 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:11.735155 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e47a703-71f9-4811-9b2a-da64c94d6100 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:11.735931 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 310/614] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:50:11.736658 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e47a703-71f9-4811-9b2a-da64c94d6100 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:11.757216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 305/615] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:11.764151 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b2cc8-dd5b-4c85-b997-340fb20bb5b0 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:11.765981 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e47a703-71f9-4811-9b2a-da64c94d6100 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:11.999022 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e47a703-71f9-4811-9b2a-da64c94d6100 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:12.062324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b2cc8-dd5b-4c85-b997-340fb20bb5b0 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:12.093217 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683f0de-650f-420f-a6bd-c17105762e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:12.094287 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683f0de-650f-420f-a6bd-c17105762e96 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:12.094780 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683f0de-650f-420f-a6bd-c17105762e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:12.095687 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683f0de-650f-420f-a6bd-c17105762e96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:12.126993 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea6d10-7075-43d7-9763-edf9cc814d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:12.128007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea6d10-7075-43d7-9763-edf9cc814d59 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:12.128504 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea6d10-7075-43d7-9763-edf9cc814d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:12.129036 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea6d10-7075-43d7-9763-edf9cc814d59 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:12.138551 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f683f0de-650f-420f-a6bd-c17105762e96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:12.139141 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea6d10-7075-43d7-9763-edf9cc814d59 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:12.145979 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea6d10-7075-43d7-9763-edf9cc814d59 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:12.153521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-f683f0de-650f-420f-a6bd-c17105762e96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:12.153521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:12.153521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:12.153521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:12.153521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:12.153521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:12.153521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:12.153521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:12.153521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:12.153521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:12.153521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:12.153521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:12.153521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:12.153521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:12.153521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:12.153521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:12.154609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:12.154609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:12.154609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:12.154609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:12.154609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:12.154609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:12.154609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:12.154609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:12.154609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:12.154609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:12.154609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:12.154609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:12.154609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:12.154609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:12.154609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:12.154609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:12.155456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:12.155456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:12.155456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:12.155456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:12.155456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:12.155456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:12.155456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:12.155456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:12.155456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:12.155456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:12.155456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:12.155456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:12.156661 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 306/616] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Aug 5 19:50:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:12.163471 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683f0de-650f-420f-a6bd-c17105762e96 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:12.165273 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-d1ea6d10-7075-43d7-9763-edf9cc814d59 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:50:12.165273 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:12.165273 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:12.165273 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:12.165273 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:12.165273 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:12.165273 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:12.165273 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:12.165273 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:12.165273 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:12.165273 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:12.165273 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:12.165273 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:12.165273 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:12.165273 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:12.165273 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:12.166362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:12.166362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:12.166362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:12.166362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:12.166362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:12.166362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:12.166362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:12.166362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:12.166362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:12.166362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:12.166362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:12.166362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:12.166362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:12.166362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:12.166362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:12.166362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:12.166362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:12.166362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:12.167409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:12.167409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:12.167409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:12.167409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:12.167409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:12.167409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:12.167409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:12.167409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:12.167409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:50:12.167409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:12.175790 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 311/617] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:50:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 407 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:12.186676 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea6d10-7075-43d7-9763-edf9cc814d59 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:12.238960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:12.293051 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:12.407312 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea6d10-7075-43d7-9763-edf9cc814d59 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:12.411051 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683f0de-650f-420f-a6bd-c17105762e96 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:12.514699 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb89d7ff-7566-4b52-8d6e-63dc44d2d08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:12.516648 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89d7ff-7566-4b52-8d6e-63dc44d2d08e None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:12.517009 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89d7ff-7566-4b52-8d6e-63dc44d2d08e None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:12.517322 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89d7ff-7566-4b52-8d6e-63dc44d2d08e None admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:12.518356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb89d7ff-7566-4b52-8d6e-63dc44d2d08e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:12.518720 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb89d7ff-7566-4b52-8d6e-63dc44d2d08e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:12.520608 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb89d7ff-7566-4b52-8d6e-63dc44d2d08e None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:12.530347 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc30236-a164-406f-8840-ca8db3d965bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:12.531476 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc30236-a164-406f-8840-ca8db3d965bf None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:12.531804 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc30236-a164-406f-8840-ca8db3d965bf None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:12.532191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc30236-a164-406f-8840-ca8db3d965bf None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:12.533383 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc30236-a164-406f-8840-ca8db3d965bf None admin] RBAC: Adding query filter params (name=service) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:12.533699 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc30236-a164-406f-8840-ca8db3d965bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:12.536300 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc30236-a164-406f-8840-ca8db3d965bf None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:12.548634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 307/618] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Aug 5 19:50:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:12.552561 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 312/619] 10.222.0.39 () {56 vars in 1135 bytes} [Fri Aug 5 19:50:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:12.553728 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc30236-a164-406f-8840-ca8db3d965bf None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:12.561673 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb89d7ff-7566-4b52-8d6e-63dc44d2d08e None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:12.744909 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb89d7ff-7566-4b52-8d6e-63dc44d2d08e None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:12.754996 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:12.777212 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:12.795641 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc30236-a164-406f-8840-ca8db3d965bf None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:12.832090 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc03b8-2da6-46cb-832f-acb9d4a7fc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:12.833086 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc03b8-2da6-46cb-832f-acb9d4a7fc04 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:12.833522 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc03b8-2da6-46cb-832f-acb9d4a7fc04 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:12.833945 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc03b8-2da6-46cb-832f-acb9d4a7fc04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:12.834782 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c345d-b75f-46d2-9f35-090dc749917c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:12.835821 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c345d-b75f-46d2-9f35-090dc749917c None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:12.836255 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c345d-b75f-46d2-9f35-090dc749917c None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:12.836673 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c345d-b75f-46d2-9f35-090dc749917c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:12.850349 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c345d-b75f-46d2-9f35-090dc749917c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:12.852096 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c345d-b75f-46d2-9f35-090dc749917c None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:12.856394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cc03b8-2da6-46cb-832f-acb9d4a7fc04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:12.864616 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cc03b8-2da6-46cb-832f-acb9d4a7fc04 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:12.868794 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-1c0c345d-b75f-46d2-9f35-090dc749917c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:12.868794 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:12.868794 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:12.868794 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:12.868794 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:12.868794 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:12.868794 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:12.868794 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:12.868794 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:12.868794 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:12.868794 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:12.868794 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:12.868794 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:12.868794 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:12.868794 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:12.869973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:12.869973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:12.869973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:12.869973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:12.869973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:12.869973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:12.869973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:12.869973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:12.869973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:12.869973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:12.869973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:12.869973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:12.869973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:12.869973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:12.869973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:12.869973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:12.869973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:12.871138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:12.871138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:12.871138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:12.871138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:12.873619 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 308/620] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:12.884608 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c345d-b75f-46d2-9f35-090dc749917c None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:12.904607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-d3cc03b8-2da6-46cb-832f-acb9d4a7fc04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:12.904607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:12.904607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:12.904607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:12.904607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:12.904607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:12.904607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:12.904607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:12.904607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:12.904607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:12.904607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:12.904607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:12.904607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:12.904607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:12.904607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:12.905719 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:12.905719 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:12.905719 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:12.905719 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:12.905719 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:12.905719 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:12.905719 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:12.905719 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:12.905719 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:12.905719 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:12.905719 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:12.905719 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:12.905719 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:12.905719 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:12.905719 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:12.905719 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:12.905719 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:12.906734 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:12.906734 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:12.906734 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:12.906734 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:12.908801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 313/621] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:12.915503 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc03b8-2da6-46cb-832f-acb9d4a7fc04 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:12.994171 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c345d-b75f-46d2-9f35-090dc749917c None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:13.044091 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eaac00-4e22-4680-a470-f9b15fd8b2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:13.045057 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaac00-4e22-4680-a470-f9b15fd8b2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:13.045614 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaac00-4e22-4680-a470-f9b15fd8b2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:13.046113 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaac00-4e22-4680-a470-f9b15fd8b2a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:13.047119 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eaac00-4e22-4680-a470-f9b15fd8b2a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:13.047433 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eaac00-4e22-4680-a470-f9b15fd8b2a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:13.048839 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eaac00-4e22-4680-a470-f9b15fd8b2a8 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:13.063731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc03b8-2da6-46cb-832f-acb9d4a7fc04 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:13.085167 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 309/622] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:13.086334 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eaac00-4e22-4680-a470-f9b15fd8b2a8 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:13.096967 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:13.125305 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb331a9f-b854-49e0-9e0a-95eb9be27492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:13.126337 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb331a9f-b854-49e0-9e0a-95eb9be27492 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:13.126827 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb331a9f-b854-49e0-9e0a-95eb9be27492 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:13.127303 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb331a9f-b854-49e0-9e0a-95eb9be27492 None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:13.128208 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb331a9f-b854-49e0-9e0a-95eb9be27492 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:13.128772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb331a9f-b854-49e0-9e0a-95eb9be27492 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:13.130394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb331a9f-b854-49e0-9e0a-95eb9be27492 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:13.160802 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 314/623] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:13.170327 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb331a9f-b854-49e0-9e0a-95eb9be27492 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:13.190486 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:13.202438 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eaac00-4e22-4680-a470-f9b15fd8b2a8 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:13.239023 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7465e-dde3-4df6-ab42-32356afa9121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:13.239659 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7465e-dde3-4df6-ab42-32356afa9121 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:13.239949 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7465e-dde3-4df6-ab42-32356afa9121 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:13.240216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7465e-dde3-4df6-ab42-32356afa9121 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:13.248683 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7465e-dde3-4df6-ab42-32356afa9121 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:13.250134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7465e-dde3-4df6-ab42-32356afa9121 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:13.262960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-aba7465e-dde3-4df6-ab42-32356afa9121 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:50:13.262960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:13.262960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:13.262960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:13.262960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:13.262960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:13.262960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:13.262960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:13.262960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:13.262960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:13.262960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:13.262960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:13.262960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:13.262960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:13.262960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:13.262960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:13.264073 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:13.264073 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:13.264073 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:13.264073 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:13.264073 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:13.264073 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:13.264073 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:13.264073 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:13.264073 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:13.264073 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:13.264073 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:13.264073 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:13.264073 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:13.264073 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:13.264073 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:13.264073 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:13.264073 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:13.264073 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:13.265071 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:13.265071 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:13.265071 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:13.265071 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:13.265071 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:13.265071 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:13.265071 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:13.265071 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:13.265071 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:50:13.265071 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:13.268654 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 310/624] 10.222.0.39 () {56 vars in 1093 bytes} [Fri Aug 5 19:50:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:13.270331 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7465e-dde3-4df6-ab42-32356afa9121 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:13.287616 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb331a9f-b854-49e0-9e0a-95eb9be27492 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:13.337903 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ec219-157c-4fca-83d0-4c7c4ab87b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:13.338910 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ec219-157c-4fca-83d0-4c7c4ab87b5e None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:13.339399 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ec219-157c-4fca-83d0-4c7c4ab87b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:13.340178 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ec219-157c-4fca-83d0-4c7c4ab87b5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:13.351556 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06ec219-157c-4fca-83d0-4c7c4ab87b5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:13.361094 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:13.361555 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06ec219-157c-4fca-83d0-4c7c4ab87b5e None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:13.388843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-d06ec219-157c-4fca-83d0-4c7c4ab87b5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:13.388843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:13.388843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:13.388843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:13.388843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:13.388843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:13.388843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:13.388843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:13.388843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:13.388843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:13.388843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:13.388843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:13.388843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:13.388843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:13.388843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:13.390016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:13.390016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:13.390016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:13.390016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:13.390016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:13.390016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:13.390016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:13.390016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:13.390016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:13.390016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:13.390016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:13.390016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:13.390016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:13.390016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:13.390016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:13.390016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:13.390016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:13.390985 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:13.390985 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:13.390985 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:13.390985 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:13.390985 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:13.390985 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:13.390985 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:13.390985 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:13.392629 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 315/625] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:13.396017 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ec219-157c-4fca-83d0-4c7c4ab87b5e None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:13.421407 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7465e-dde3-4df6-ab42-32356afa9121 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:13.462146 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ada6dee-72c7-465f-96c7-93e7077acb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:13.462757 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada6dee-72c7-465f-96c7-93e7077acb74 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:13.463033 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada6dee-72c7-465f-96c7-93e7077acb74 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:13.463304 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada6dee-72c7-465f-96c7-93e7077acb74 None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:13.464121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ada6dee-72c7-465f-96c7-93e7077acb74 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:13.464425 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ada6dee-72c7-465f-96c7-93e7077acb74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:13.466717 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ada6dee-72c7-465f-96c7-93e7077acb74 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:13.484810 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 311/626] 10.222.0.39 () {56 vars in 1133 bytes} [Fri Aug 5 19:50:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:13.486060 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9bb6d-e0ab-4376-9064-6df33db70376 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:13.486343 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9bb6d-e0ab-4376-9064-6df33db70376 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:13.486617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9bb6d-e0ab-4376-9064-6df33db70376 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:13.492672 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 312/627] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:13.493432 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9bb6d-e0ab-4376-9064-6df33db70376 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:13.536962 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:13.567646 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ec219-157c-4fca-83d0-4c7c4ab87b5e None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:13.609321 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9bb6d-e0ab-4376-9064-6df33db70376 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:13.613960 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fadb6f-4476-4ffe-b41b-c0b3f945355a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:13.614580 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fadb6f-4476-4ffe-b41b-c0b3f945355a None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:13.614863 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fadb6f-4476-4ffe-b41b-c0b3f945355a None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:13.615140 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fadb6f-4476-4ffe-b41b-c0b3f945355a None admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:13.615893 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fadb6f-4476-4ffe-b41b-c0b3f945355a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:13.616202 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fadb6f-4476-4ffe-b41b-c0b3f945355a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:13.617707 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fadb6f-4476-4ffe-b41b-c0b3f945355a None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:13.630813 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:13.635832 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 316/628] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:13.637369 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810d0e2-1ac4-44d3-9842-2aeebec7773e None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:13.637548 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810d0e2-1ac4-44d3-9842-2aeebec7773e None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:13.637722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810d0e2-1ac4-44d3-9842-2aeebec7773e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:13.646722 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982daf2d-e5dd-404b-93de-28265c0f27f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:13.647706 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982daf2d-e5dd-404b-93de-28265c0f27f2 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:13.648186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982daf2d-e5dd-404b-93de-28265c0f27f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:13.648696 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982daf2d-e5dd-404b-93de-28265c0f27f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:13.656702 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-4810d0e2-1ac4-44d3-9842-2aeebec7773e None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:13.657914 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-4810d0e2-1ac4-44d3-9842-2aeebec7773e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:13.725363 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982daf2d-e5dd-404b-93de-28265c0f27f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:13.726787 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982daf2d-e5dd-404b-93de-28265c0f27f2 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:13.740772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 317/629] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:50:13.740772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4462d345-7a3d-4296-9c5e-9eff3f6b7926 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:13.740772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4462d345-7a3d-4296-9c5e-9eff3f6b7926 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:13.740772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4462d345-7a3d-4296-9c5e-9eff3f6b7926 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:13.740772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 318/630] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:13.740772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4462d345-7a3d-4296-9c5e-9eff3f6b7926 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:13.816008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:13.850931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 313/631] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:50:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:50:13.850931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe1d32-bf53-4549-8613-c2926bda5bc5 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:13.850931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe1d32-bf53-4549-8613-c2926bda5bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:13.850931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe1d32-bf53-4549-8613-c2926bda5bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:13.858990 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4462d345-7a3d-4296-9c5e-9eff3f6b7926 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:13.867892 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-9cfe1d32-bf53-4549-8613-c2926bda5bc5 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:13.869033 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-9cfe1d32-bf53-4549-8613-c2926bda5bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:13.886909 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:13.898355 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f7ee1f-5db2-4be4-9fab-c1d5ec6c86ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:13.899290 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7ee1f-5db2-4be4-9fab-c1d5ec6c86ed None admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:13.899575 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7ee1f-5db2-4be4-9fab-c1d5ec6c86ed None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:13.899858 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7ee1f-5db2-4be4-9fab-c1d5ec6c86ed None admin] PATH_INFO: `/v3/projects/80379e2104644e98a2e85d4e4af1b365/users/91d3d43cb0bd4ca3b8e8f10e29cc421c/roles/7ebfd9e58fe549f08cdaec5f906a38a6` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:13.915657 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f7ee1f-5db2-4be4-9fab-c1d5ec6c86ed None admin] RBAC: Authorizing `identity:create_grant(project_id=80379e2104644e98a2e85d4e4af1b365, user_id=91d3d43cb0bd4ca3b8e8f10e29cc421c, role_id=7ebfd9e58fe549f08cdaec5f906a38a6)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:13.917151 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f7ee1f-5db2-4be4-9fab-c1d5ec6c86ed None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:13.932333 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 319/632] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Aug 5 19:50:13 2022] PUT /identity/v3/projects/80379e2104644e98a2e85d4e4af1b365/users/91d3d43cb0bd4ca3b8e8f10e29cc421c/roles/7ebfd9e58fe549f08cdaec5f906a38a6 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:13.933178 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f7ee1f-5db2-4be4-9fab-c1d5ec6c86ed None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:13.957633 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 314/633] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:50:13.958905 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe1d32-bf53-4549-8613-c2926bda5bc5 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:14.116272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe1d32-bf53-4549-8613-c2926bda5bc5 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:14.135347 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f7ee1f-5db2-4be4-9fab-c1d5ec6c86ed None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:14.165206 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16049b-0c17-4e01-8e1c-94ff02932bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:14.165856 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16049b-0c17-4e01-8e1c-94ff02932bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:14.166147 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16049b-0c17-4e01-8e1c-94ff02932bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:14.166441 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16049b-0c17-4e01-8e1c-94ff02932bd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:14.171486 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:14.182603 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d16049b-0c17-4e01-8e1c-94ff02932bd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:14.185849 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d16049b-0c17-4e01-8e1c-94ff02932bd4 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:14.202894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-1d16049b-0c17-4e01-8e1c-94ff02932bd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:14.202894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:14.202894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:14.202894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:14.202894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:14.202894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:14.202894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:14.202894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:14.202894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:14.202894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:14.202894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:14.202894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:14.202894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:14.202894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:14.202894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:14.203950 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:14.203950 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:14.203950 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:14.203950 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:14.203950 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:14.203950 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:14.203950 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:14.203950 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:14.203950 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:14.203950 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:14.203950 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:14.203950 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:14.203950 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:14.203950 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:14.203950 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:14.203950 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:14.203950 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:14.205276 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:14.205276 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:14.205276 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:14.205276 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:14.205276 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 315/634] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:14.205995 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16049b-0c17-4e01-8e1c-94ff02932bd4 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:14.209480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:14.212617 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f4a8b7-bff7-460a-8a07-502fa8e74438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:14.212617 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4a8b7-bff7-460a-8a07-502fa8e74438 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:14.212617 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4a8b7-bff7-460a-8a07-502fa8e74438 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:14.212617 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4a8b7-bff7-460a-8a07-502fa8e74438 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:14.225257 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f4a8b7-bff7-460a-8a07-502fa8e74438 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:14.230738 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-55f4a8b7-bff7-460a-8a07-502fa8e74438 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:14.230738 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:14.230738 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:14.230738 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:14.230738 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:14.230738 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:14.230738 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:14.230738 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:14.230738 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:14.230738 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:14.230738 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:14.230738 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:14.230738 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:14.230738 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:14.230738 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:14.230738 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:14.231941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:14.231941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:14.231941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:14.231941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:14.231941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:14.231941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:14.231941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:14.231941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:14.231941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:14.231941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:14.231941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:14.231941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:14.231941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:14.231941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:14.231941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:14.231941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:14.232833 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:14.232833 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:14.232833 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:14.232833 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:14.232833 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:14.232833 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:14.232833 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:14.232833 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:14.232833 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:14.232833 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:14.232833 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:14.232833 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:14.232833 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 320/635] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Aug 5 19:50:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:14.238511 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee7bad-17b6-4152-8fcf-7dd37dbf94b0 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:14.238961 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee7bad-17b6-4152-8fcf-7dd37dbf94b0 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:14.239362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee7bad-17b6-4152-8fcf-7dd37dbf94b0 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:14.241110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 321/636] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:14.242015 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee7bad-17b6-4152-8fcf-7dd37dbf94b0 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:14.336480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16049b-0c17-4e01-8e1c-94ff02932bd4 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:14.363847 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee7bad-17b6-4152-8fcf-7dd37dbf94b0 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:14.377693 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e841c3c-119b-45ce-8464-d8d73e5f26ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:14.378416 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e841c3c-119b-45ce-8464-d8d73e5f26ab None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:14.378701 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e841c3c-119b-45ce-8464-d8d73e5f26ab None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:14.379087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e841c3c-119b-45ce-8464-d8d73e5f26ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:14.391079 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e841c3c-119b-45ce-8464-d8d73e5f26ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:14.397140 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-7e841c3c-119b-45ce-8464-d8d73e5f26ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:14.397140 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:14.397140 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:14.397140 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:14.397140 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:14.397140 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:14.397140 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:14.397140 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:14.397140 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:14.397140 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:14.397140 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:14.397140 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:14.397140 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:14.397140 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:14.397140 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:14.397140 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:14.398116 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:14.398116 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:14.398116 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:14.398116 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:14.398116 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:14.398116 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:14.398116 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:14.398116 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:14.398116 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:14.398116 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:14.398116 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:14.398116 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:14.398116 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:14.398116 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:14.398116 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:14.398116 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:14.399254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:14.399254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:14.399254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:14.399254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:14.399254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:14.399254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:14.399254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:14.399254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:14.399254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:14.399254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:14.399254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:14.399254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:14.399254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 316/637] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Aug 5 19:50:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:14.399961 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e841c3c-119b-45ce-8464-d8d73e5f26ab None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:14.407858 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46181b97-3b41-4d88-a0fb-6fed477f867a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:14.408493 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46181b97-3b41-4d88-a0fb-6fed477f867a None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:14.408806 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46181b97-3b41-4d88-a0fb-6fed477f867a None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:14.409137 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46181b97-3b41-4d88-a0fb-6fed477f867a None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:14.409843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46181b97-3b41-4d88-a0fb-6fed477f867a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:14.410145 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46181b97-3b41-4d88-a0fb-6fed477f867a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:14.411595 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46181b97-3b41-4d88-a0fb-6fed477f867a None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:14.429878 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 322/638] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:14.429878 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6cbab-ffec-4003-be45-470a26eb2636 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:14.429878 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6cbab-ffec-4003-be45-470a26eb2636 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:14.429878 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6cbab-ffec-4003-be45-470a26eb2636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:14.447914 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-3fd6cbab-ffec-4003-be45-470a26eb2636 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:14.449148 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-3fd6cbab-ffec-4003-be45-470a26eb2636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:14.475779 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:14.485192 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:14.525442 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e841c3c-119b-45ce-8464-d8d73e5f26ab None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:14.526938 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 323/639] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:50:14.528031 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd6cbab-ffec-4003-be45-470a26eb2636 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:14.576475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5430928a-2c46-45ea-8c43-1f4f04d853d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:14.577120 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5430928a-2c46-45ea-8c43-1f4f04d853d8 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:14.577407 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5430928a-2c46-45ea-8c43-1f4f04d853d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:14.577678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5430928a-2c46-45ea-8c43-1f4f04d853d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:14.578403 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5430928a-2c46-45ea-8c43-1f4f04d853d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:14.578708 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5430928a-2c46-45ea-8c43-1f4f04d853d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:14.580074 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5430928a-2c46-45ea-8c43-1f4f04d853d8 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:14.588930 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 317/640] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Aug 5 19:50:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:14.589747 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5430928a-2c46-45ea-8c43-1f4f04d853d8 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:14.645796 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd6cbab-ffec-4003-be45-470a26eb2636 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:14.691834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d64f79e-1070-42a8-8f89-1bb0c6b2d059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:14.692454 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64f79e-1070-42a8-8f89-1bb0c6b2d059 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:14.692759 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64f79e-1070-42a8-8f89-1bb0c6b2d059 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:14.693033 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64f79e-1070-42a8-8f89-1bb0c6b2d059 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:14.693750 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d64f79e-1070-42a8-8f89-1bb0c6b2d059 None admin] RBAC: Adding query filter params (name=service) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:14.694050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d64f79e-1070-42a8-8f89-1bb0c6b2d059 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:14.695437 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d64f79e-1070-42a8-8f89-1bb0c6b2d059 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:14.704778 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 324/641] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Aug 5 19:50:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:14.710639 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d64f79e-1070-42a8-8f89-1bb0c6b2d059 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:14.748986 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5430928a-2c46-45ea-8c43-1f4f04d853d8 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:14.754895 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:14.787854 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:14.803328 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e88fc-85ca-4407-8ece-63c9269a5f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:14.804520 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e88fc-85ca-4407-8ece-63c9269a5f87 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:14.805109 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e88fc-85ca-4407-8ece-63c9269a5f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:14.805737 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e88fc-85ca-4407-8ece-63c9269a5f87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:14.814743 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1e88fc-85ca-4407-8ece-63c9269a5f87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:14.816597 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1e88fc-85ca-4407-8ece-63c9269a5f87 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:14.830336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-3c1e88fc-85ca-4407-8ece-63c9269a5f87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:14.830336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:14.830336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:14.830336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:14.830336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:14.830336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:14.830336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:14.830336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:14.830336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:14.830336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:14.830336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:14.830336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:14.830336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:14.830336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:14.830336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:14.837159 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:14.837159 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:14.837159 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:14.837159 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:14.837159 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:14.837159 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:14.837159 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:14.837159 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:14.837159 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:14.837159 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:14.837159 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:14.837159 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:14.837159 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:14.837159 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:14.837159 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:14.837159 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:14.837159 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:14.838195 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:14.838195 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:14.838195 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:14.838195 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:14.838195 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:14.838195 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:14.838195 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:14.838195 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:14.838195 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 318/642] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:14.838195 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e88fc-85ca-4407-8ece-63c9269a5f87 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:14.852453 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d64f79e-1070-42a8-8f89-1bb0c6b2d059 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:14.894672 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988aaa07-6623-4a83-9f6d-e167ba1888ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:14.895635 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988aaa07-6623-4a83-9f6d-e167ba1888ec None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:14.895962 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988aaa07-6623-4a83-9f6d-e167ba1888ec None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:14.896175 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988aaa07-6623-4a83-9f6d-e167ba1888ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:14.907989 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988aaa07-6623-4a83-9f6d-e167ba1888ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:14.913361 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-988aaa07-6623-4a83-9f6d-e167ba1888ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:14.913361 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:14.913361 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:14.913361 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:14.913361 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:14.913361 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:14.913361 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:14.913361 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:14.913361 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:14.913361 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:14.913361 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:14.913361 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:14.913361 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:14.913361 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:14.913361 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:14.913361 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:14.914394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:14.914394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:14.914394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:14.914394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:14.914394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:14.914394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:14.914394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:14.914394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:14.914394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:14.914394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:14.914394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:14.914394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:14.914394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:14.914394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:14.914394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:14.914394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:14.915244 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:14.915244 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:14.915244 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:14.915244 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:14.915244 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:14.915244 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:14.915244 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:14.915244 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:14.915244 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:14.915244 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:14.915244 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:14.915244 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:14.915244 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 325/643] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Aug 5 19:50:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:14.916596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988aaa07-6623-4a83-9f6d-e167ba1888ec None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:14.957157 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e88fc-85ca-4407-8ece-63c9269a5f87 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:14.998868 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569cceb-10c9-4774-9772-83599da9278b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:14.999631 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569cceb-10c9-4774-9772-83599da9278b None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:14.999916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569cceb-10c9-4774-9772-83599da9278b None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:15.000193 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569cceb-10c9-4774-9772-83599da9278b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:15.016345 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:15.016824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3569cceb-10c9-4774-9772-83599da9278b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:15.018360 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3569cceb-10c9-4774-9772-83599da9278b None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:15.029204 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988aaa07-6623-4a83-9f6d-e167ba1888ec None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:15.035232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-3569cceb-10c9-4774-9772-83599da9278b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:15.035232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:15.035232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:15.035232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:15.035232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:15.035232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:15.035232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:15.035232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:15.035232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:15.035232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:15.035232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:15.035232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:15.035232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:15.035232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:15.035232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:15.036158 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:15.036158 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:15.036158 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:15.036158 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:15.036158 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:15.036158 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:15.036158 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:15.036158 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:15.036158 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:15.036158 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:15.036158 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:15.036158 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:15.036158 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:15.036158 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:15.036158 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:15.036158 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:15.036158 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:15.042387 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:15.042387 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:15.042387 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:15.042387 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:15.042387 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 319/644] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:15.042387 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569cceb-10c9-4774-9772-83599da9278b None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:15.073021 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac83f07-bea1-487c-a1cf-7ea486263742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:15.073654 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac83f07-bea1-487c-a1cf-7ea486263742 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:15.073931 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac83f07-bea1-487c-a1cf-7ea486263742 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:15.074200 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac83f07-bea1-487c-a1cf-7ea486263742 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:15.079780 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:15.085906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac83f07-bea1-487c-a1cf-7ea486263742 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:15.087378 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac83f07-bea1-487c-a1cf-7ea486263742 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:15.102897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-dac83f07-bea1-487c-a1cf-7ea486263742 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:15.102897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:15.102897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:15.102897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:15.102897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:15.102897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:15.102897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:15.102897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:15.102897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:15.102897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:15.102897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:15.102897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:15.102897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:15.102897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:15.102897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:15.104012 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:15.104012 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:15.104012 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:15.104012 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:15.104012 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:15.104012 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:15.104012 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:15.104012 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:15.104012 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:15.104012 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:15.104012 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:15.104012 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:15.104012 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:15.104012 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:15.104012 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:15.104012 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:15.104012 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:15.105164 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:15.105164 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:15.105164 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:15.105164 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:15.108664 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 326/645] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:15.109600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac83f07-bea1-487c-a1cf-7ea486263742 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:15.164636 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569cceb-10c9-4774-9772-83599da9278b None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:15.204785 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827fb11-b501-4281-abc8-93bdb3a2cfe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:15.205390 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827fb11-b501-4281-abc8-93bdb3a2cfe0 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:15.205682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827fb11-b501-4281-abc8-93bdb3a2cfe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:15.205955 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827fb11-b501-4281-abc8-93bdb3a2cfe0 None admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:15.206712 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827fb11-b501-4281-abc8-93bdb3a2cfe0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:15.207018 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827fb11-b501-4281-abc8-93bdb3a2cfe0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:15.208448 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827fb11-b501-4281-abc8-93bdb3a2cfe0 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:15.222575 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 320/646] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:15.223210 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827fb11-b501-4281-abc8-93bdb3a2cfe0 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:15.230630 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac83f07-bea1-487c-a1cf-7ea486263742 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:15.278112 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dafed5-8570-4f39-b361-68c8f2fbb2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:15.278704 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dafed5-8570-4f39-b361-68c8f2fbb2fc None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:15.278922 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dafed5-8570-4f39-b361-68c8f2fbb2fc None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:15.279273 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dafed5-8570-4f39-b361-68c8f2fbb2fc None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:15.280007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dafed5-8570-4f39-b361-68c8f2fbb2fc None admin] RBAC: Adding query filter params (name=service) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:15.280314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dafed5-8570-4f39-b361-68c8f2fbb2fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:15.281735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dafed5-8570-4f39-b361-68c8f2fbb2fc None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:15.292620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 327/647] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Aug 5 19:50:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:15.292620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dafed5-8570-4f39-b361-68c8f2fbb2fc None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:15.296245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:15.336261 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:15.343943 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827fb11-b501-4281-abc8-93bdb3a2cfe0 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:15.383650 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54f004-e748-4f03-a421-42f856cd5a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:15.384359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54f004-e748-4f03-a421-42f856cd5a52 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:15.384779 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54f004-e748-4f03-a421-42f856cd5a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:15.385071 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54f004-e748-4f03-a421-42f856cd5a52 None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:15.385776 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54f004-e748-4f03-a421-42f856cd5a52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:15.386082 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54f004-e748-4f03-a421-42f856cd5a52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:15.387770 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54f004-e748-4f03-a421-42f856cd5a52 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:15.401515 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 321/648] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:15.402273 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54f004-e748-4f03-a421-42f856cd5a52 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:15.402596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dafed5-8570-4f39-b361-68c8f2fbb2fc None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:15.443361 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b86c7-8162-47ea-9aba-c7e1fe0cacec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:15.443991 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b86c7-8162-47ea-9aba-c7e1fe0cacec None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:15.444267 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b86c7-8162-47ea-9aba-c7e1fe0cacec None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:15.444838 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b86c7-8162-47ea-9aba-c7e1fe0cacec None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:15.445265 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b86c7-8162-47ea-9aba-c7e1fe0cacec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:15.445571 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b86c7-8162-47ea-9aba-c7e1fe0cacec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:15.446992 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b86c7-8162-47ea-9aba-c7e1fe0cacec None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:15.462068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 328/649] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:15.462937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b86c7-8162-47ea-9aba-c7e1fe0cacec None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:15.545137 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:15.553182 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54f004-e748-4f03-a421-42f856cd5a52 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:15.591535 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b86c7-8162-47ea-9aba-c7e1fe0cacec None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:15.603406 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c09b8-cefa-40f3-bc98-ba2b3b6b0751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:15.604027 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c09b8-cefa-40f3-bc98-ba2b3b6b0751 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:15.604306 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c09b8-cefa-40f3-bc98-ba2b3b6b0751 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:15.604594 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c09b8-cefa-40f3-bc98-ba2b3b6b0751 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:15.606612 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58c09b8-cefa-40f3-bc98-ba2b3b6b0751 None admin] RBAC: Adding query filter params (role.id=7ebfd9e58fe549f08cdaec5f906a38a6, scope.project.id=80379e2104644e98a2e85d4e4af1b365, user.id=95e9f085e4f04d118373b8abff27543a) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:15.607198 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58c09b8-cefa-40f3-bc98-ba2b3b6b0751 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:15.609084 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58c09b8-cefa-40f3-bc98-ba2b3b6b0751 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:15.624967 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 322/650] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:50:15 2022] GET /identity/v3/role_assignments?user.id=95e9f085e4f04d118373b8abff27543a&scope.project.id=80379e2104644e98a2e85d4e4af1b365&role.id=7ebfd9e58fe549f08cdaec5f906a38a6 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:15.624967 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c09b8-cefa-40f3-bc98-ba2b3b6b0751 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:15.648230 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d9cca-c2f3-4ee9-be93-b522828ad943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:15.649752 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d9cca-c2f3-4ee9-be93-b522828ad943 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:15.650212 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d9cca-c2f3-4ee9-be93-b522828ad943 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:15.650661 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d9cca-c2f3-4ee9-be93-b522828ad943 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:15.653658 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:15.666390 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d9cca-c2f3-4ee9-be93-b522828ad943 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:15.668095 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d9cca-c2f3-4ee9-be93-b522828ad943 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:15.691313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-704d9cca-c2f3-4ee9-be93-b522828ad943 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:15.691313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:15.691313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:15.691313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:15.691313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:15.691313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:15.691313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:15.691313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:15.691313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:15.691313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:15.691313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:15.691313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:15.691313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:15.691313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:15.691313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:15.692268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:15.692268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:15.692268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:15.692268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:15.692268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:15.692268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:15.692268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:15.692268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:15.692268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:15.692268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:15.692268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:15.692268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:15.692268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:15.692268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:15.692268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:15.692268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:15.692268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:15.693209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:15.693209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:15.693209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:15.693209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:15.693209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 329/651] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:15.694376 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d9cca-c2f3-4ee9-be93-b522828ad943 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:15.786463 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c09b8-cefa-40f3-bc98-ba2b3b6b0751 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:15.830255 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d9cca-c2f3-4ee9-be93-b522828ad943 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:15.837617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f2ff2-af58-4087-ba4c-eb2c799a20ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:15.838452 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f2ff2-af58-4087-ba4c-eb2c799a20ae None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:15.838582 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f2ff2-af58-4087-ba4c-eb2c799a20ae None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:15.838801 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f2ff2-af58-4087-ba4c-eb2c799a20ae None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:15.848901 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874f2ff2-af58-4087-ba4c-eb2c799a20ae None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:15.850001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:15.850519 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874f2ff2-af58-4087-ba4c-eb2c799a20ae None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:15.864510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-874f2ff2-af58-4087-ba4c-eb2c799a20ae None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 19:50:15.864510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:15.864510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:15.864510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:15.864510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:15.864510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:15.864510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:15.864510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:15.864510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:15.864510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:15.864510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:15.864510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:15.864510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:15.864510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:15.864510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:15.864510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:15.865555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:15.865555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:15.865555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:15.865555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:15.865555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:15.865555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:15.865555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:15.865555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:15.865555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:15.865555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:15.865555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:15.865555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:15.865555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:15.865555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:15.865555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:15.865555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:15.865555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:15.865555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:15.866678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:15.866678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:15.866678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:15.866678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:15.866678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:15.866678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:15.866678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:15.866678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:15.866678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 19:50:15.866678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:15.867222 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 323/652] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:50:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:15.868000 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f2ff2-af58-4087-ba4c-eb2c799a20ae None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:15.877808 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e85cc-a2bd-492d-a911-c32a8e2338e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:15.878458 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e85cc-a2bd-492d-a911-c32a8e2338e9 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:15.878743 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e85cc-a2bd-492d-a911-c32a8e2338e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:15.879019 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e85cc-a2bd-492d-a911-c32a8e2338e9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:15.890114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e85cc-a2bd-492d-a911-c32a8e2338e9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:15.891699 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e85cc-a2bd-492d-a911-c32a8e2338e9 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:15.906834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-dd5e85cc-a2bd-492d-a911-c32a8e2338e9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 05 19:50:15.906834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:15.906834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:15.906834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:15.906834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:15.906834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:15.906834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:15.906834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:15.906834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:15.906834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:15.906834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:15.906834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:15.906834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:15.906834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:15.906834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:15.906834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:15.908026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:15.908026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:15.908026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:15.908026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:15.908026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:15.908026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:15.908026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:15.908026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:15.908026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:15.908026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:15.908026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:15.908026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:15.908026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:15.908026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:15.908026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:15.908026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:15.908026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:15.908026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:15.909136 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:15.909136 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:15.909136 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:15.909136 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:15.909136 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:15.909136 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:15.909136 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:15.909136 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:15.909136 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 05 19:50:15.909136 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:15.909701 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 330/653] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:50:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:15.910325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e85cc-a2bd-492d-a911-c32a8e2338e9 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:15.978310 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:16.010368 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f2ff2-af58-4087-ba4c-eb2c799a20ae None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:16.042204 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e85cc-a2bd-492d-a911-c32a8e2338e9 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:16.053835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e579b55-bcd9-4160-ae03-6ecdd8f5e8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:16.055025 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e579b55-bcd9-4160-ae03-6ecdd8f5e8af None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:16.055518 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e579b55-bcd9-4160-ae03-6ecdd8f5e8af None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:16.055996 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e579b55-bcd9-4160-ae03-6ecdd8f5e8af None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:16.056955 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e579b55-bcd9-4160-ae03-6ecdd8f5e8af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:16.057572 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e579b55-bcd9-4160-ae03-6ecdd8f5e8af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:16.059973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e579b55-bcd9-4160-ae03-6ecdd8f5e8af None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:16.074880 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 324/654] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:16.080680 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e579b55-bcd9-4160-ae03-6ecdd8f5e8af None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:16.086072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b74812-2ca9-4ade-8f96-e2e914143687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:16.086072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b74812-2ca9-4ade-8f96-e2e914143687 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:16.086498 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b74812-2ca9-4ade-8f96-e2e914143687 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:16.086665 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b74812-2ca9-4ade-8f96-e2e914143687 None admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:16.087536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b74812-2ca9-4ade-8f96-e2e914143687 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:16.087845 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b74812-2ca9-4ade-8f96-e2e914143687 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:16.089514 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b74812-2ca9-4ade-8f96-e2e914143687 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:16.111082 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 331/655] 10.222.0.39 () {56 vars in 1137 bytes} [Fri Aug 5 19:50:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:16.111954 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b74812-2ca9-4ade-8f96-e2e914143687 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:16.162191 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:16.223061 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e579b55-bcd9-4160-ae03-6ecdd8f5e8af None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:16.266304 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b74812-2ca9-4ade-8f96-e2e914143687 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:16.272382 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997462b9-6601-4a96-9f7a-db20eada7758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:16.273042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997462b9-6601-4a96-9f7a-db20eada7758 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:16.273324 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997462b9-6601-4a96-9f7a-db20eada7758 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:16.273595 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997462b9-6601-4a96-9f7a-db20eada7758 None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:16.275126 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997462b9-6601-4a96-9f7a-db20eada7758 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:16.275454 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997462b9-6601-4a96-9f7a-db20eada7758 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:16.277297 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997462b9-6601-4a96-9f7a-db20eada7758 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:16.288687 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:16.303051 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 325/656] 10.222.0.39 () {56 vars in 1137 bytes} [Fri Aug 5 19:50:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:16.303051 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997462b9-6601-4a96-9f7a-db20eada7758 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:16.313334 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6931fb1c-4b06-49e2-9f69-75e8e423ad99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:16.313976 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6931fb1c-4b06-49e2-9f69-75e8e423ad99 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:16.314256 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6931fb1c-4b06-49e2-9f69-75e8e423ad99 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:16.314529 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6931fb1c-4b06-49e2-9f69-75e8e423ad99 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:16.325488 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6931fb1c-4b06-49e2-9f69-75e8e423ad99 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:16.327098 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6931fb1c-4b06-49e2-9f69-75e8e423ad99 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:16.340483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-6931fb1c-4b06-49e2-9f69-75e8e423ad99 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 19:50:16.340483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:16.340483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:16.340483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:16.340483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:16.340483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:16.340483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:16.340483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:16.340483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:16.340483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:16.340483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:16.340483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:16.340483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:16.340483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:16.340483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:16.340483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:16.341538 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:16.341538 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:16.341538 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:16.341538 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:16.341538 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:16.341538 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:16.341538 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:16.341538 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:16.341538 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:16.341538 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:16.341538 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:16.341538 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:16.341538 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:16.341538 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:16.341538 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:16.341538 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:16.341538 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:16.341538 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:16.342483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:16.342483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:16.342483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:16.342483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:16.342483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:16.342483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:16.342483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:16.342483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:16.342483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 19:50:16.342483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:16.342483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 332/657] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Aug 5 19:50:16 2022] GET /identity/v3/users/neutron => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:16.343088 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669126a-b911-4d7b-b49b-686888004886 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:16.343703 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669126a-b911-4d7b-b49b-686888004886 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:16.343703 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669126a-b911-4d7b-b49b-686888004886 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:16.348767 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 333/658] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:16.348767 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669126a-b911-4d7b-b49b-686888004886 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:16.416562 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997462b9-6601-4a96-9f7a-db20eada7758 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:16.457049 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c81ebd-ceae-4a5c-ab47-55b42304a579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:16.457665 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c81ebd-ceae-4a5c-ab47-55b42304a579 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:16.457960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c81ebd-ceae-4a5c-ab47-55b42304a579 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:16.458224 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c81ebd-ceae-4a5c-ab47-55b42304a579 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:16.462960 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:16.472459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669126a-b911-4d7b-b49b-686888004886 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:16.509512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707235b9-a26d-4597-bfe3-0ff7f68bc64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:16.510134 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707235b9-a26d-4597-bfe3-0ff7f68bc64c None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:16.510418 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707235b9-a26d-4597-bfe3-0ff7f68bc64c None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:16.510695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707235b9-a26d-4597-bfe3-0ff7f68bc64c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:16.539645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:16.545067 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c81ebd-ceae-4a5c-ab47-55b42304a579 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:16.547216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c81ebd-ceae-4a5c-ab47-55b42304a579 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:16.599936 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707235b9-a26d-4597-bfe3-0ff7f68bc64c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:16.601902 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707235b9-a26d-4597-bfe3-0ff7f68bc64c None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:16.672761 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 326/659] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:50:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:50:16.673898 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c81ebd-ceae-4a5c-ab47-55b42304a579 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:16.713172 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:16.722031 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 334/660] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:50:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:50:16.723245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab0fa5-68bb-4946-b65c-19cdf58520b9 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:16.723534 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab0fa5-68bb-4946-b65c-19cdf58520b9 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:16.723807 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab0fa5-68bb-4946-b65c-19cdf58520b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:16.742312 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-f4ab0fa5-68bb-4946-b65c-19cdf58520b9 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:16.743625 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-f4ab0fa5-68bb-4946-b65c-19cdf58520b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:16.786116 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:16.790629 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c81ebd-ceae-4a5c-ab47-55b42304a579 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:16.819136 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 335/661] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:50:16.820333 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab0fa5-68bb-4946-b65c-19cdf58520b9 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:16.845000 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a695f-8fde-4208-a1a3-abd42180f521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:16.845736 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a695f-8fde-4208-a1a3-abd42180f521 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:16.846022 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a695f-8fde-4208-a1a3-abd42180f521 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:16.846300 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a695f-8fde-4208-a1a3-abd42180f521 None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:16.847225 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a695f-8fde-4208-a1a3-abd42180f521 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:16.847535 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a695f-8fde-4208-a1a3-abd42180f521 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:16.849010 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a695f-8fde-4208-a1a3-abd42180f521 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:16.873879 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 327/662] 10.222.0.39 () {56 vars in 1139 bytes} [Fri Aug 5 19:50:16 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:16.875675 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a695f-8fde-4208-a1a3-abd42180f521 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:16.940555 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab0fa5-68bb-4946-b65c-19cdf58520b9 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:16.979299 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:16.995104 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a695f-8fde-4208-a1a3-abd42180f521 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:16.995737 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dbc479-ae8c-44fe-a002-e4562f66abd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:16.997313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dbc479-ae8c-44fe-a002-e4562f66abd3 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:16.997761 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dbc479-ae8c-44fe-a002-e4562f66abd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:16.998127 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dbc479-ae8c-44fe-a002-e4562f66abd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:17.010786 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dbc479-ae8c-44fe-a002-e4562f66abd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:17.012324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dbc479-ae8c-44fe-a002-e4562f66abd3 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:17.030929 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-82dbc479-ae8c-44fe-a002-e4562f66abd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:17.030929 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:17.030929 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:17.030929 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:17.030929 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:17.030929 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:17.030929 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:17.030929 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:17.030929 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:17.030929 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:17.030929 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:17.030929 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:17.030929 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:17.030929 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:17.030929 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:17.031923 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:17.031923 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:17.031923 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:17.031923 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:17.031923 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:17.031923 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:17.031923 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:17.031923 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:17.031923 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:17.031923 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:17.031923 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:17.031923 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:17.031923 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:17.031923 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:17.031923 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:17.031923 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:17.031923 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:17.033277 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:17.033277 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:17.033277 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:17.033277 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:17.033277 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 336/663] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:17.033277 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dbc479-ae8c-44fe-a002-e4562f66abd3 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:17.043143 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c7a768-c86a-4696-818e-7376f7db4dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:17.044335 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7a768-c86a-4696-818e-7376f7db4dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:17.044630 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7a768-c86a-4696-818e-7376f7db4dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:17.044899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7a768-c86a-4696-818e-7376f7db4dc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:17.055751 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c7a768-c86a-4696-818e-7376f7db4dc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:17.057251 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c7a768-c86a-4696-818e-7376f7db4dc7 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:17.073946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-a6c7a768-c86a-4696-818e-7376f7db4dc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:17.073946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:17.073946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:17.073946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:17.073946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:17.073946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:17.073946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:17.073946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:17.073946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:17.073946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:17.073946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:17.073946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:17.073946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:17.073946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:17.073946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:17.075004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:17.075004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:17.075004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:17.075004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:17.075004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:17.075004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:17.075004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:17.075004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:17.075004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:17.075004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:17.075004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:17.075004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:17.075004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:17.075004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:17.075004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:17.075004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:17.075004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:17.075956 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:17.075956 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:17.075956 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:17.075956 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:17.075956 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 328/664] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:17.076476 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c7a768-c86a-4696-818e-7376f7db4dc7 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:17.094346 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:17.160858 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dbc479-ae8c-44fe-a002-e4562f66abd3 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:17.196902 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d28f91-801d-4393-b2f4-413027eeb762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:17.197509 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d28f91-801d-4393-b2f4-413027eeb762 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:17.197867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d28f91-801d-4393-b2f4-413027eeb762 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:17.198148 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d28f91-801d-4393-b2f4-413027eeb762 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:17.206086 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c7a768-c86a-4696-818e-7376f7db4dc7 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:17.209470 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d28f91-801d-4393-b2f4-413027eeb762 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:17.215590 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-d8d28f91-801d-4393-b2f4-413027eeb762 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:17.215590 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:17.215590 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:17.215590 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:17.215590 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:17.215590 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:17.215590 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:17.215590 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:17.215590 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:17.215590 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:17.215590 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:17.215590 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:17.215590 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:17.215590 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:17.215590 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:17.215590 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:17.217283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:17.217283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:17.217283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:17.217283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:17.217283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:17.217283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:17.217283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:17.217283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:17.217283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:17.217283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:17.217283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:17.217283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:17.217283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:17.217283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:17.217283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:17.217283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:17.218122 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:17.218122 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:17.218122 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:17.218122 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:17.218122 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:17.218122 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:17.218122 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:17.218122 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:17.218122 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:17.218122 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:17.218122 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:17.218122 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:17.218122 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 337/665] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Aug 5 19:50:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:17.218122 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d28f91-801d-4393-b2f4-413027eeb762 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:17.251975 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e09278-29df-4129-a80b-a03bae1f00d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:17.252615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e09278-29df-4129-a80b-a03bae1f00d6 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:17.252901 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e09278-29df-4129-a80b-a03bae1f00d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:17.253174 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e09278-29df-4129-a80b-a03bae1f00d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:17.267619 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:17.333764 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e09278-29df-4129-a80b-a03bae1f00d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:17.335279 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e09278-29df-4129-a80b-a03bae1f00d6 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:17.345453 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d28f91-801d-4393-b2f4-413027eeb762 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:17.346021 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:17.384763 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8076fae7-8558-445f-ad44-7badb4d729cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:17.385737 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076fae7-8558-445f-ad44-7badb4d729cc None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:17.386164 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076fae7-8558-445f-ad44-7badb4d729cc None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:17.386646 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076fae7-8558-445f-ad44-7badb4d729cc None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:17.388261 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8076fae7-8558-445f-ad44-7badb4d729cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:17.388760 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8076fae7-8558-445f-ad44-7badb4d729cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:17.390398 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8076fae7-8558-445f-ad44-7badb4d729cc None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:17.404743 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 338/666] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:17.405752 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8076fae7-8558-445f-ad44-7badb4d729cc None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:17.452693 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 329/667] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:50:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:50:17.452693 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e09278-29df-4129-a80b-a03bae1f00d6 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:17.521587 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:17.529703 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8076fae7-8558-445f-ad44-7badb4d729cc None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:17.568800 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e4594b-330d-4427-88e8-4401acae53ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:17.569427 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4594b-330d-4427-88e8-4401acae53ac None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:17.569708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4594b-330d-4427-88e8-4401acae53ac None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:17.569986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4594b-330d-4427-88e8-4401acae53ac None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:17.570687 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e4594b-330d-4427-88e8-4401acae53ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:17.570990 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e4594b-330d-4427-88e8-4401acae53ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:17.572431 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e4594b-330d-4427-88e8-4401acae53ac None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:17.577404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e09278-29df-4129-a80b-a03bae1f00d6 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:17.591586 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 339/668] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:17.591586 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e4594b-330d-4427-88e8-4401acae53ac None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:17.592866 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:17.619209 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd3ff2-df35-4637-863b-6fedb5a9302b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:17.619826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd3ff2-df35-4637-863b-6fedb5a9302b None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:17.620096 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd3ff2-df35-4637-863b-6fedb5a9302b None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:17.620428 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd3ff2-df35-4637-863b-6fedb5a9302b None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:17.621473 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd3ff2-df35-4637-863b-6fedb5a9302b None admin] RBAC: Adding query filter params (name=service) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:17.621584 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd3ff2-df35-4637-863b-6fedb5a9302b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:17.623114 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd3ff2-df35-4637-863b-6fedb5a9302b None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:17.632493 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 330/669] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Aug 5 19:50:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:17.638034 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd3ff2-df35-4637-863b-6fedb5a9302b None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:17.723970 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e4594b-330d-4427-88e8-4401acae53ac None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:17.764082 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd3ff2-df35-4637-863b-6fedb5a9302b None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:17.775603 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7234f7b-1e98-4b1b-999b-36befbad5569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:17.776889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7234f7b-1e98-4b1b-999b-36befbad5569 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:17.777369 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7234f7b-1e98-4b1b-999b-36befbad5569 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:17.778263 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7234f7b-1e98-4b1b-999b-36befbad5569 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:17.788289 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7234f7b-1e98-4b1b-999b-36befbad5569 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:17.791034 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7234f7b-1e98-4b1b-999b-36befbad5569 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:17.800850 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c074a12-d2b2-4c57-9449-192e2cd4c4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:17.801492 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c074a12-d2b2-4c57-9449-192e2cd4c4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:17.801774 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c074a12-d2b2-4c57-9449-192e2cd4c4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:17.802049 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c074a12-d2b2-4c57-9449-192e2cd4c4e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:17.808207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-b7234f7b-1e98-4b1b-999b-36befbad5569 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:17.808207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:17.808207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:17.808207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:17.808207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:17.808207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:17.808207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:17.808207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:17.808207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:17.808207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:17.808207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:17.808207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:17.808207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:17.808207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:17.808207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:17.809731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:17.809731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:17.809731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:17.809731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:17.809731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:17.809731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:17.809731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:17.809731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:17.809731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:17.809731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:17.809731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:17.809731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:17.809731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:17.809731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:17.809731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:17.809731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:17.809731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:17.810598 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:17.810598 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:17.810598 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:17.810598 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:17.810598 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:17.810598 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:17.810598 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:17.810598 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:17.812655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 340/670] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:17.813660 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7234f7b-1e98-4b1b-999b-36befbad5569 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:17.822312 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c074a12-d2b2-4c57-9449-192e2cd4c4e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:17.824662 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c074a12-d2b2-4c57-9449-192e2cd4c4e2 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:17.840008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:17.841202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-9c074a12-d2b2-4c57-9449-192e2cd4c4e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:17.841202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:17.841202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:17.841202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:17.841202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:17.841202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:17.841202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:17.841202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:17.841202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:17.841202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:17.841202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:17.841202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:17.841202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:17.841202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:17.841202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:17.842106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:17.842106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:17.842106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:17.842106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:17.842106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:17.842106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:17.842106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:17.842106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:17.842106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:17.842106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:17.842106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:17.842106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:17.842106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:17.842106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:17.842106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:17.842106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:17.842106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:17.843034 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:17.843034 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:17.843034 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:17.843034 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:17.843034 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 331/671] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:17.843034 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c074a12-d2b2-4c57-9449-192e2cd4c4e2 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:17.877257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:17.933839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7234f7b-1e98-4b1b-999b-36befbad5569 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:17.968091 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8bbb82-371b-4e82-a5e9-b26baa8cdcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:17.968757 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8bbb82-371b-4e82-a5e9-b26baa8cdcfd None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:17.969035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8bbb82-371b-4e82-a5e9-b26baa8cdcfd None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:17.969311 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8bbb82-371b-4e82-a5e9-b26baa8cdcfd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:17.977240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8bbb82-371b-4e82-a5e9-b26baa8cdcfd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:17.978399 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8bbb82-371b-4e82-a5e9-b26baa8cdcfd None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:17.991012 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c074a12-d2b2-4c57-9449-192e2cd4c4e2 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:17.991202 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-ef8bbb82-371b-4e82-a5e9-b26baa8cdcfd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:17.991202 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:17.991202 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:17.991202 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:17.991202 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:17.991202 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:17.991202 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:17.991202 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:17.991202 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:17.991202 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:17.991202 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:17.991202 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:17.991202 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:17.991202 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:17.991202 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:17.992163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:17.992163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:17.992163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:17.992163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:17.992163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:17.992163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:17.992163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:17.992163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:17.992163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:17.992163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:17.992163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:17.992163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:17.992163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:17.992163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:17.992163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:17.992163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:17.992163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:17.993236 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:17.993236 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:17.993236 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:17.993236 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:17.993236 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:17.993236 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:17.993236 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:17.993236 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:17.993236 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 341/672] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:17.993757 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8bbb82-371b-4e82-a5e9-b26baa8cdcfd None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:18.028093 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e5c84-76ac-4c05-a431-25ab766fe63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:18.028768 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e5c84-76ac-4c05-a431-25ab766fe63f None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:18.029041 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e5c84-76ac-4c05-a431-25ab766fe63f None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:18.029311 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e5c84-76ac-4c05-a431-25ab766fe63f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:18.040161 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e5c84-76ac-4c05-a431-25ab766fe63f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:18.041640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e5c84-76ac-4c05-a431-25ab766fe63f None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:18.056873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-3e8e5c84-76ac-4c05-a431-25ab766fe63f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:18.056873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:18.056873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:18.056873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:18.056873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:18.056873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:18.056873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:18.056873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:18.056873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:18.056873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:18.056873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:18.056873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:18.056873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:18.056873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:18.056873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:18.057845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:18.057845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:18.057845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:18.057845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:18.057845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:18.057845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:18.057845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:18.057845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:18.057845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:18.057845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:18.057845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:18.057845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:18.057845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:18.057845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:18.057845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:18.057845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:18.057845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:18.058744 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:18.058744 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:18.058744 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:18.058744 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:18.058744 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 332/673] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:18.059647 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e5c84-76ac-4c05-a431-25ab766fe63f None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:18.078407 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:18.103946 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8bbb82-371b-4e82-a5e9-b26baa8cdcfd None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:18.145097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b0f54-24d6-4299-baf8-ef5d5a9da85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:18.145724 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b0f54-24d6-4299-baf8-ef5d5a9da85c None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:18.146005 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b0f54-24d6-4299-baf8-ef5d5a9da85c None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:18.146281 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b0f54-24d6-4299-baf8-ef5d5a9da85c None admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:18.146480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:18.147124 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092b0f54-24d6-4299-baf8-ef5d5a9da85c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:18.147558 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092b0f54-24d6-4299-baf8-ef5d5a9da85c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:18.149311 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092b0f54-24d6-4299-baf8-ef5d5a9da85c None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:18.162581 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 342/674] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:18.163987 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139df94-d4aa-4172-947f-0e7404b0850e None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:18.164270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139df94-d4aa-4172-947f-0e7404b0850e None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:18.164550 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139df94-d4aa-4172-947f-0e7404b0850e None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:18.165979 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 343/675] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:18.169103 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139df94-d4aa-4172-947f-0e7404b0850e None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:18.179062 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e5c84-76ac-4c05-a431-25ab766fe63f None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:18.214801 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee4661-d88f-4d02-a394-cd620e4ebb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:18.216021 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee4661-d88f-4d02-a394-cd620e4ebb08 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:18.216335 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee4661-d88f-4d02-a394-cd620e4ebb08 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:18.216563 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee4661-d88f-4d02-a394-cd620e4ebb08 None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:18.217268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee4661-d88f-4d02-a394-cd620e4ebb08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:18.217572 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee4661-d88f-4d02-a394-cd620e4ebb08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:18.218931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee4661-d88f-4d02-a394-cd620e4ebb08 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:18.232498 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 333/676] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:18.233666 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee4661-d88f-4d02-a394-cd620e4ebb08 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:18.280701 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139df94-d4aa-4172-947f-0e7404b0850e None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:18.323243 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe513f7-e5dd-40d7-87a7-0d795a4e8d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:18.323866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe513f7-e5dd-40d7-87a7-0d795a4e8d39 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:18.324148 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe513f7-e5dd-40d7-87a7-0d795a4e8d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:18.324976 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe513f7-e5dd-40d7-87a7-0d795a4e8d39 None admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:18.325745 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe513f7-e5dd-40d7-87a7-0d795a4e8d39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:18.326049 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe513f7-e5dd-40d7-87a7-0d795a4e8d39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:18.327494 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe513f7-e5dd-40d7-87a7-0d795a4e8d39 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:18.330283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:18.338468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee4661-d88f-4d02-a394-cd620e4ebb08 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:18.341987 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 344/677] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:18.342915 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe513f7-e5dd-40d7-87a7-0d795a4e8d39 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:18.374758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053a9b3-cd18-4c2a-bc36-2d1ba8ee930c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:18.375372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053a9b3-cd18-4c2a-bc36-2d1ba8ee930c None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:18.375649 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053a9b3-cd18-4c2a-bc36-2d1ba8ee930c None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:18.375927 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053a9b3-cd18-4c2a-bc36-2d1ba8ee930c None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:18.376645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e053a9b3-cd18-4c2a-bc36-2d1ba8ee930c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:18.376957 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e053a9b3-cd18-4c2a-bc36-2d1ba8ee930c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:18.378310 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e053a9b3-cd18-4c2a-bc36-2d1ba8ee930c None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:18.379778 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:18.391883 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 334/678] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:18.393119 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa078b9-b0ba-4830-9c6c-b1102be499cc None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:18.393409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa078b9-b0ba-4830-9c6c-b1102be499cc None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:18.393682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa078b9-b0ba-4830-9c6c-b1102be499cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:18.412364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-8aa078b9-b0ba-4830-9c6c-b1102be499cc None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:18.413592 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-8aa078b9-b0ba-4830-9c6c-b1102be499cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:18.458269 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe513f7-e5dd-40d7-87a7-0d795a4e8d39 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:18.484135 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 335/679] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:50:18.484949 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa078b9-b0ba-4830-9c6c-b1102be499cc None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:18.494747 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43614a95-654f-479d-b7c0-9b2551e9a707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:18.495372 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43614a95-654f-479d-b7c0-9b2551e9a707 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:18.495654 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43614a95-654f-479d-b7c0-9b2551e9a707 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:18.496357 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43614a95-654f-479d-b7c0-9b2551e9a707 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:18.498048 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43614a95-654f-479d-b7c0-9b2551e9a707 None admin] RBAC: Adding query filter params (role.id=7ebfd9e58fe549f08cdaec5f906a38a6, scope.project.id=80379e2104644e98a2e85d4e4af1b365, user.id=44dfe45766d743558ed82e464721cf72) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:18.498707 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43614a95-654f-479d-b7c0-9b2551e9a707 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:18.500170 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43614a95-654f-479d-b7c0-9b2551e9a707 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:18.510104 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 345/680] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:50:18 2022] GET /identity/v3/role_assignments?user.id=44dfe45766d743558ed82e464721cf72&scope.project.id=80379e2104644e98a2e85d4e4af1b365&role.id=7ebfd9e58fe549f08cdaec5f906a38a6 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:18.510104 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43614a95-654f-479d-b7c0-9b2551e9a707 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:18.608719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa078b9-b0ba-4830-9c6c-b1102be499cc None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:18.611479 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:18.637819 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:18.650011 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e823a5e-5fd4-4a24-b742-1b2a0aa5af66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:18.650663 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e823a5e-5fd4-4a24-b742-1b2a0aa5af66 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:18.650944 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e823a5e-5fd4-4a24-b742-1b2a0aa5af66 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:18.651221 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e823a5e-5fd4-4a24-b742-1b2a0aa5af66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:18.659312 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e823a5e-5fd4-4a24-b742-1b2a0aa5af66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:18.660806 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e823a5e-5fd4-4a24-b742-1b2a0aa5af66 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:18.667580 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43614a95-654f-479d-b7c0-9b2551e9a707 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:18.674882 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-2e823a5e-5fd4-4a24-b742-1b2a0aa5af66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:18.674882 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:18.674882 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:18.674882 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:18.674882 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:18.674882 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:18.674882 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:18.674882 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:18.674882 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:18.674882 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:18.674882 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:18.674882 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:18.674882 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:18.674882 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:18.674882 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:18.675805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:18.675805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:18.675805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:18.675805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:18.675805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:18.675805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:18.675805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:18.675805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:18.675805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:18.675805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:18.675805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:18.675805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:18.675805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:18.675805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:18.675805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:18.675805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:18.675805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:18.677694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:18.677694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:18.677694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:18.677694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:18.677694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:18.677694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:18.677694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:18.677694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:18.677694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 336/681] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:18.677694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e823a5e-5fd4-4a24-b742-1b2a0aa5af66 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:18.710736 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496377b3-e439-4ed5-a99f-29c1900948b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:18.711688 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496377b3-e439-4ed5-a99f-29c1900948b1 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:18.711688 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496377b3-e439-4ed5-a99f-29c1900948b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:18.711900 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496377b3-e439-4ed5-a99f-29c1900948b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:18.712670 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496377b3-e439-4ed5-a99f-29c1900948b1 None admin] RBAC: Adding query filter params (role.id=7ebfd9e58fe549f08cdaec5f906a38a6, scope.project.id=80379e2104644e98a2e85d4e4af1b365, user.id=2fe69b398c1f4e18b625a63ae40b01e8) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:18.712980 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496377b3-e439-4ed5-a99f-29c1900948b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:18.714412 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496377b3-e439-4ed5-a99f-29c1900948b1 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:18.721773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 346/682] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:50:18 2022] GET /identity/v3/role_assignments?user.id=2fe69b398c1f4e18b625a63ae40b01e8&scope.project.id=80379e2104644e98a2e85d4e4af1b365&role.id=7ebfd9e58fe549f08cdaec5f906a38a6 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:18.724672 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496377b3-e439-4ed5-a99f-29c1900948b1 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:18.817102 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e823a5e-5fd4-4a24-b742-1b2a0aa5af66 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:18.856425 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496377b3-e439-4ed5-a99f-29c1900948b1 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:18.882051 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e11bf4-11e5-4629-9764-db63fcd56ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:18.883106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e11bf4-11e5-4629-9764-db63fcd56ed5 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:18.883695 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e11bf4-11e5-4629-9764-db63fcd56ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:18.884178 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e11bf4-11e5-4629-9764-db63fcd56ed5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:18.885550 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:18.903461 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398d5dc0-1918-4c4a-a8cf-71bef7c18461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:18.904614 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d5dc0-1918-4c4a-a8cf-71bef7c18461 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:18.904987 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d5dc0-1918-4c4a-a8cf-71bef7c18461 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:18.905329 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d5dc0-1918-4c4a-a8cf-71bef7c18461 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:18.906251 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e11bf4-11e5-4629-9764-db63fcd56ed5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:18.908649 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e11bf4-11e5-4629-9764-db63fcd56ed5 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:18.919345 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398d5dc0-1918-4c4a-a8cf-71bef7c18461 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:18.924939 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-398d5dc0-1918-4c4a-a8cf-71bef7c18461 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:50:18.924939 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:18.924939 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:18.924939 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:18.924939 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:18.924939 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:18.924939 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:18.924939 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:18.924939 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:18.924939 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:18.924939 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:18.924939 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:18.924939 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:18.924939 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:18.924939 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:18.924939 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:18.925899 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:18.925899 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:18.925899 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:18.925899 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:18.925899 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:18.925899 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:18.925899 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:18.925899 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:18.925899 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:18.925899 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:18.925899 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:18.925899 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:18.925899 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:18.925899 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:18.925899 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:18.925899 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:18.926744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:18.926744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:18.926744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:18.926744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:18.926744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:18.926744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:18.926744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:18.926744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:18.926744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:18.926744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:18.926744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:50:18.926744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:18.926744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 347/683] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:50:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:18.927434 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398d5dc0-1918-4c4a-a8cf-71bef7c18461 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:18.932919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-24e11bf4-11e5-4629-9764-db63fcd56ed5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:50:18.932919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:18.932919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:18.932919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:18.932919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:18.932919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:18.932919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:18.932919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:18.932919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:18.932919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:18.932919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:18.932919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:18.932919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:18.932919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:18.932919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:18.932919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:18.933931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:18.933931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:18.933931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:18.933931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:18.933931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:18.933931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:18.933931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:18.933931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:18.933931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:18.933931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:18.933931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:18.933931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:18.933931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:18.933931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:18.933931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:18.933931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:18.933931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:18.933931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:18.934885 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:18.934885 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:18.934885 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:18.934885 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:18.934885 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:18.934885 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:18.934885 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:18.934885 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:18.934885 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:50:18.934885 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:18.935928 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 337/684] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:50:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:18.937693 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e11bf4-11e5-4629-9764-db63fcd56ed5 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:19.014912 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:19.084750 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e11bf4-11e5-4629-9764-db63fcd56ed5 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:19.086795 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398d5dc0-1918-4c4a-a8cf-71bef7c18461 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:19.133205 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6a500-aa03-480a-9cc3-6dcd8eb8ac83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:19.134490 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6a500-aa03-480a-9cc3-6dcd8eb8ac83 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:19.134993 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6a500-aa03-480a-9cc3-6dcd8eb8ac83 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:19.135490 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6a500-aa03-480a-9cc3-6dcd8eb8ac83 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:19.136772 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d6a500-aa03-480a-9cc3-6dcd8eb8ac83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:19.137866 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d6a500-aa03-480a-9cc3-6dcd8eb8ac83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:19.139051 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22281a2-095f-432f-9d87-ad573cb2a03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:19.139848 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22281a2-095f-432f-9d87-ad573cb2a03e None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:19.140284 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22281a2-095f-432f-9d87-ad573cb2a03e None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:19.140734 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22281a2-095f-432f-9d87-ad573cb2a03e None admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:19.141312 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d6a500-aa03-480a-9cc3-6dcd8eb8ac83 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:19.141774 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22281a2-095f-432f-9d87-ad573cb2a03e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:19.142235 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22281a2-095f-432f-9d87-ad573cb2a03e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:19.143808 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22281a2-095f-432f-9d87-ad573cb2a03e None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:19.160272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 338/685] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:50:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:19.161736 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 348/686] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:19.161877 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6a500-aa03-480a-9cc3-6dcd8eb8ac83 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:19.165503 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22281a2-095f-432f-9d87-ad573cb2a03e None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:19.237035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:19.296386 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22281a2-095f-432f-9d87-ad573cb2a03e None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:19.339265 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b3e07-4a55-4668-808c-9321aa7bd711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:19.340659 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b3e07-4a55-4668-808c-9321aa7bd711 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:19.341102 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b3e07-4a55-4668-808c-9321aa7bd711 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:19.341988 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b3e07-4a55-4668-808c-9321aa7bd711 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:19.351555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6a500-aa03-480a-9cc3-6dcd8eb8ac83 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:19.355429 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b3e07-4a55-4668-808c-9321aa7bd711 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:19.358650 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b3e07-4a55-4668-808c-9321aa7bd711 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:19.366686 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:19.378776 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-de7b3e07-4a55-4668-808c-9321aa7bd711 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:19.378776 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:19.378776 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:19.378776 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:19.378776 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:19.378776 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:19.378776 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:19.378776 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:19.378776 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:19.378776 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:19.378776 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:19.378776 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:19.378776 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:19.378776 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:19.378776 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:19.379950 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:19.379950 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:19.379950 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:19.379950 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:19.379950 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:19.379950 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:19.379950 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:19.379950 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:19.379950 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:19.379950 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:19.379950 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:19.379950 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:19.379950 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:19.379950 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:19.379950 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:19.379950 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:19.379950 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:19.381046 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:19.381046 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:19.381046 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:19.381046 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:19.382966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 349/687] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:19.388925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b3e07-4a55-4668-808c-9321aa7bd711 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:19.417491 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a6de2-194a-453f-acbc-0da63fa03089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:19.418945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a6de2-194a-453f-acbc-0da63fa03089 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:19.419411 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a6de2-194a-453f-acbc-0da63fa03089 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:19.419863 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a6de2-194a-453f-acbc-0da63fa03089 None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:19.421231 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36a6de2-194a-453f-acbc-0da63fa03089 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:19.421751 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36a6de2-194a-453f-acbc-0da63fa03089 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:19.423837 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36a6de2-194a-453f-acbc-0da63fa03089 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:19.447825 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 339/688] 10.222.0.39 () {56 vars in 1135 bytes} [Fri Aug 5 19:50:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:19.448977 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a6de2-194a-453f-acbc-0da63fa03089 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:19.561382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:19.568289 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a6de2-194a-453f-acbc-0da63fa03089 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:19.573292 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b3e07-4a55-4668-808c-9321aa7bd711 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:19.608096 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a0fba-2a53-4b6e-b83e-a1b10d537c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:19.609040 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a0fba-2a53-4b6e-b83e-a1b10d537c37 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:19.609505 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a0fba-2a53-4b6e-b83e-a1b10d537c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:19.609961 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a0fba-2a53-4b6e-b83e-a1b10d537c37 None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:19.610859 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a0fba-2a53-4b6e-b83e-a1b10d537c37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:19.611345 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a0fba-2a53-4b6e-b83e-a1b10d537c37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:19.613128 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a0fba-2a53-4b6e-b83e-a1b10d537c37 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:19.631393 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 340/689] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:19.632606 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a0fba-2a53-4b6e-b83e-a1b10d537c37 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:19.639216 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc2a377-359b-4873-95de-4ed443b87e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:19.640662 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2a377-359b-4873-95de-4ed443b87e40 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:19.641245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2a377-359b-4873-95de-4ed443b87e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:19.641823 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2a377-359b-4873-95de-4ed443b87e40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:19.643481 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc2a377-359b-4873-95de-4ed443b87e40 None admin] RBAC: Adding query filter params (role.id=7ebfd9e58fe549f08cdaec5f906a38a6, scope.project.id=80379e2104644e98a2e85d4e4af1b365, user.id=b1c75368c00e4b0cb7b3163728843479) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:19.644180 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc2a377-359b-4873-95de-4ed443b87e40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:19.647140 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc2a377-359b-4873-95de-4ed443b87e40 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:19.659062 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 350/690] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:50:19 2022] GET /identity/v3/role_assignments?user.id=b1c75368c00e4b0cb7b3163728843479&scope.project.id=80379e2104644e98a2e85d4e4af1b365&role.id=7ebfd9e58fe549f08cdaec5f906a38a6 => generated 608 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:19.664089 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:19.666862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc2a377-359b-4873-95de-4ed443b87e40 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:19.758095 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a0fba-2a53-4b6e-b83e-a1b10d537c37 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:19.844161 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c81d4-e2cd-413a-ad88-ca640bc2dd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:19.845875 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c81d4-e2cd-413a-ad88-ca640bc2dd77 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:19.846899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c81d4-e2cd-413a-ad88-ca640bc2dd77 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:19.847332 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c81d4-e2cd-413a-ad88-ca640bc2dd77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:19.850484 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc2a377-359b-4873-95de-4ed443b87e40 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:19.914690 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d778a5-13f9-46e6-ac41-6f5b02e70cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:19.915338 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d778a5-13f9-46e6-ac41-6f5b02e70cbb None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:19.915625 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d778a5-13f9-46e6-ac41-6f5b02e70cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:19.915893 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d778a5-13f9-46e6-ac41-6f5b02e70cbb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:19.925180 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d778a5-13f9-46e6-ac41-6f5b02e70cbb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:19.926641 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d778a5-13f9-46e6-ac41-6f5b02e70cbb None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:19.940744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-04d778a5-13f9-46e6-ac41-6f5b02e70cbb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:50:19.940744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:19.940744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:19.940744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:19.940744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:19.940744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:19.940744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:19.940744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:19.940744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:19.940744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:19.940744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:19.940744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:19.940744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:19.940744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:19.940744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:19.940744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:19.941826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:19.941826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:19.941826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:19.941826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:19.941826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:19.941826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:19.941826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:19.941826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:19.941826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:19.941826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:19.941826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:19.941826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:19.941826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:19.941826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:19.941826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:19.941826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:19.941826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:19.941826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:19.942755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:19.942755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:19.942755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:19.942755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:19.942755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:19.942755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:19.942755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:19.942755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:19.942755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:50:19.942755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:19.942755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 351/691] 10.222.0.39 () {56 vars in 1093 bytes} [Fri Aug 5 19:50:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:19.944859 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1c81d4-e2cd-413a-ad88-ca640bc2dd77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:19.946395 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1c81d4-e2cd-413a-ad88-ca640bc2dd77 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:19.950407 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d778a5-13f9-46e6-ac41-6f5b02e70cbb None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:19.980320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:19.988950 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:20.074171 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 341/692] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:50:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:50:20.080412 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d778a5-13f9-46e6-ac41-6f5b02e70cbb None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:20.083609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c81d4-e2cd-413a-ad88-ca640bc2dd77 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:20.131334 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ab084-2b44-49b9-9794-490505be3553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:20.132090 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ab084-2b44-49b9-9794-490505be3553 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:20.132364 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ab084-2b44-49b9-9794-490505be3553 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:20.132679 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ab084-2b44-49b9-9794-490505be3553 None admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:20.133478 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ab084-2b44-49b9-9794-490505be3553 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:20.133783 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ab084-2b44-49b9-9794-490505be3553 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:20.135441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ab084-2b44-49b9-9794-490505be3553 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:20.156660 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 352/693] 10.222.0.39 () {56 vars in 1133 bytes} [Fri Aug 5 19:50:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:20.161787 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ab084-2b44-49b9-9794-490505be3553 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:20.203587 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c81d4-e2cd-413a-ad88-ca640bc2dd77 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:20.232618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:20.246842 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c4f5f-6728-4c15-a628-7363bb92d5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:20.247901 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c4f5f-6728-4c15-a628-7363bb92d5b7 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:20.248254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c4f5f-6728-4c15-a628-7363bb92d5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:20.248571 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c4f5f-6728-4c15-a628-7363bb92d5b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:20.256237 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:20.262459 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c4f5f-6728-4c15-a628-7363bb92d5b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:20.264051 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c4f5f-6728-4c15-a628-7363bb92d5b7 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:20.281542 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ab084-2b44-49b9-9794-490505be3553 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:20.283243 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-be9c4f5f-6728-4c15-a628-7363bb92d5b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:20.283243 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:20.283243 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:20.283243 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:20.283243 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:20.283243 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:20.283243 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:20.283243 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:20.283243 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:20.283243 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:20.283243 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:20.283243 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:20.283243 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:20.283243 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:20.283243 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:20.284103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:20.284103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:20.284103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:20.284103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:20.284103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:20.284103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:20.284103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:20.284103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:20.284103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:20.284103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:20.284103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:20.284103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:20.284103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:20.284103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:20.284103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:20.284103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:20.284103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:20.285049 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:20.285049 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:20.285049 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:20.285049 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:20.288988 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 342/694] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:20.291416 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c4f5f-6728-4c15-a628-7363bb92d5b7 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:20.320960 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f403d25-13f4-476a-b8a1-29951b121dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:20.321674 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f403d25-13f4-476a-b8a1-29951b121dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:20.321958 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f403d25-13f4-476a-b8a1-29951b121dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:20.322228 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f403d25-13f4-476a-b8a1-29951b121dd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:20.397525 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f403d25-13f4-476a-b8a1-29951b121dd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:20.399338 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f403d25-13f4-476a-b8a1-29951b121dd7 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:20.406620 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c4f5f-6728-4c15-a628-7363bb92d5b7 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:20.444962 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2969dff-d2a6-4c26-9a20-6cbfbf2b3eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:20.445678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2969dff-d2a6-4c26-9a20-6cbfbf2b3eed None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:20.445962 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2969dff-d2a6-4c26-9a20-6cbfbf2b3eed None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:20.446243 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2969dff-d2a6-4c26-9a20-6cbfbf2b3eed None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:20.447042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2969dff-d2a6-4c26-9a20-6cbfbf2b3eed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:20.447352 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2969dff-d2a6-4c26-9a20-6cbfbf2b3eed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:20.448736 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2969dff-d2a6-4c26-9a20-6cbfbf2b3eed None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:20.462882 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 343/695] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:20.469268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2969dff-d2a6-4c26-9a20-6cbfbf2b3eed None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:20.487173 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:20.510256 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:20.514914 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 353/696] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:50:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:50:20.526255 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f403d25-13f4-476a-b8a1-29951b121dd7 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:20.591104 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2969dff-d2a6-4c26-9a20-6cbfbf2b3eed None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:20.631534 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e886cb-90fe-4841-94d4-96680acc9692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:20.633444 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e886cb-90fe-4841-94d4-96680acc9692 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:20.633444 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e886cb-90fe-4841-94d4-96680acc9692 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:20.633444 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e886cb-90fe-4841-94d4-96680acc9692 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:20.642912 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e886cb-90fe-4841-94d4-96680acc9692 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:20.644294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e886cb-90fe-4841-94d4-96680acc9692 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:20.658354 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-d3e886cb-90fe-4841-94d4-96680acc9692 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:20.658354 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:20.658354 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:20.658354 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:20.658354 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:20.658354 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:20.658354 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:20.658354 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:20.658354 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:20.658354 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:20.658354 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:20.658354 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:20.658354 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:20.658354 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:20.658354 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:20.660456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:20.660456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:20.660456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:20.660456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:20.660456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:20.660456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:20.660456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:20.660456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:20.660456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:20.660456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:20.660456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:20.660456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:20.660456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:20.660456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:20.660456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:20.660456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:20.660456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:20.662608 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:20.662608 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:20.662608 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:20.662608 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:20.662608 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:20.662608 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:20.662608 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:20.662608 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:20.662608 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 344/697] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:20.666542 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e886cb-90fe-4841-94d4-96680acc9692 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:20.773139 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:20.780968 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e886cb-90fe-4841-94d4-96680acc9692 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:20.818924 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f403d25-13f4-476a-b8a1-29951b121dd7 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:20.820865 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f65ec-3912-455c-97b1-1565c36f9461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:20.821550 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f65ec-3912-455c-97b1-1565c36f9461 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:20.821834 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f65ec-3912-455c-97b1-1565c36f9461 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:20.822113 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f65ec-3912-455c-97b1-1565c36f9461 None admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:20.822875 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243f65ec-3912-455c-97b1-1565c36f9461 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:20.823179 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243f65ec-3912-455c-97b1-1565c36f9461 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:20.824867 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243f65ec-3912-455c-97b1-1565c36f9461 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:20.839106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 345/698] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:20.844998 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f65ec-3912-455c-97b1-1565c36f9461 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:20.896843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87304ab9-1a7c-4dfb-8a60-83c518cda93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:20.897869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87304ab9-1a7c-4dfb-8a60-83c518cda93e None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:20.898424 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87304ab9-1a7c-4dfb-8a60-83c518cda93e None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:20.899150 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87304ab9-1a7c-4dfb-8a60-83c518cda93e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:20.922358 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87304ab9-1a7c-4dfb-8a60-83c518cda93e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:20.923875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87304ab9-1a7c-4dfb-8a60-83c518cda93e None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:20.944016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-87304ab9-1a7c-4dfb-8a60-83c518cda93e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:20.944016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:20.944016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:20.944016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:20.944016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:20.944016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:20.944016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:20.944016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:20.944016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:20.944016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:20.944016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:20.944016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:20.944016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:20.944016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:20.944016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:20.945038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:20.945038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:20.945038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:20.945038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:20.945038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:20.945038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:20.945038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:20.945038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:20.945038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:20.945038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:20.945038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:20.945038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:20.945038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:20.945038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:20.945038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:20.945038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:20.945038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:20.945936 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:20.945936 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:20.945936 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:20.945936 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:20.949987 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 354/699] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 420 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:20.951291 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87304ab9-1a7c-4dfb-8a60-83c518cda93e None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:20.963666 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f65ec-3912-455c-97b1-1565c36f9461 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:20.984489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:21.005044 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6637f7-27fa-4a75-ba1d-9b6efe58e2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:21.005758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6637f7-27fa-4a75-ba1d-9b6efe58e2aa None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:21.006041 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6637f7-27fa-4a75-ba1d-9b6efe58e2aa None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:21.006317 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6637f7-27fa-4a75-ba1d-9b6efe58e2aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:21.007165 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6637f7-27fa-4a75-ba1d-9b6efe58e2aa None admin] RBAC: Adding query filter params (role.id=7ebfd9e58fe549f08cdaec5f906a38a6, scope.project.id=80379e2104644e98a2e85d4e4af1b365, user.id=91d3d43cb0bd4ca3b8e8f10e29cc421c) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:21.007531 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6637f7-27fa-4a75-ba1d-9b6efe58e2aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:21.009096 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6637f7-27fa-4a75-ba1d-9b6efe58e2aa None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:21.017032 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 346/700] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:50:20 2022] GET /identity/v3/role_assignments?user.id=91d3d43cb0bd4ca3b8e8f10e29cc421c&scope.project.id=80379e2104644e98a2e85d4e4af1b365&role.id=7ebfd9e58fe549f08cdaec5f906a38a6 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:21.030801 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687af35-8a9f-4731-b229-585ac5ff0062 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:21.031105 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687af35-8a9f-4731-b229-585ac5ff0062 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:21.031261 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687af35-8a9f-4731-b229-585ac5ff0062 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:21.032829 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 347/701] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:50:21.038923 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541b5d5-9f5f-4f65-965d-94e55f0ed84c None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:21.039635 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541b5d5-9f5f-4f65-965d-94e55f0ed84c None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:21.040140 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541b5d5-9f5f-4f65-965d-94e55f0ed84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:21.062243 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-c541b5d5-9f5f-4f65-965d-94e55f0ed84c None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:21.063461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-c541b5d5-9f5f-4f65-965d-94e55f0ed84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:21.070115 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87304ab9-1a7c-4dfb-8a60-83c518cda93e None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:21.080365 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:21.118331 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227818f6-988f-44cb-bd42-0e559e216a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:21.119367 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227818f6-988f-44cb-bd42-0e559e216a8e None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:21.119866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227818f6-988f-44cb-bd42-0e559e216a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:21.120359 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227818f6-988f-44cb-bd42-0e559e216a8e None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:21.121854 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227818f6-988f-44cb-bd42-0e559e216a8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:21.122894 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227818f6-988f-44cb-bd42-0e559e216a8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:21.124553 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227818f6-988f-44cb-bd42-0e559e216a8e None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:21.154251 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 355/702] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:21.161136 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227818f6-988f-44cb-bd42-0e559e216a8e None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:21.193292 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 348/703] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:50:21.223416 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541b5d5-9f5f-4f65-965d-94e55f0ed84c None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:21.300802 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:21.315982 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227818f6-988f-44cb-bd42-0e559e216a8e None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:21.361045 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d91d0-4197-4eb0-af89-5fca5d1096e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:21.362480 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d91d0-4197-4eb0-af89-5fca5d1096e9 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:21.362962 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d91d0-4197-4eb0-af89-5fca5d1096e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:21.363429 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d91d0-4197-4eb0-af89-5fca5d1096e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:21.372647 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d91d0-4197-4eb0-af89-5fca5d1096e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:21.374677 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d91d0-4197-4eb0-af89-5fca5d1096e9 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:21.389117 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-854d91d0-4197-4eb0-af89-5fca5d1096e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:21.389117 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:21.389117 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:21.389117 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:21.389117 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:21.389117 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:21.389117 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:21.389117 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:21.389117 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:21.389117 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:21.389117 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:21.389117 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:21.389117 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:21.389117 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:21.389117 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:21.390066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:21.390066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:21.390066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:21.390066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:21.390066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:21.390066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:21.390066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:21.390066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:21.390066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:21.390066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:21.390066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:21.390066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:21.390066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:21.390066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:21.390066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:21.390066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:21.390066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:21.390954 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:21.390954 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:21.390954 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:21.390954 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:21.390954 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:21.390954 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:21.390954 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:21.390954 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:21.393532 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 356/704] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:21.395849 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55525fef-2cca-4c77-b321-f51d243f9aef None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:21.396539 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55525fef-2cca-4c77-b321-f51d243f9aef None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:21.397057 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55525fef-2cca-4c77-b321-f51d243f9aef None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:21.399199 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 357/705] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:21.400196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55525fef-2cca-4c77-b321-f51d243f9aef None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:21.415365 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541b5d5-9f5f-4f65-965d-94e55f0ed84c None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:21.464280 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ce789-4031-4c80-a0ac-86b86db920fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:21.465077 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ce789-4031-4c80-a0ac-86b86db920fd None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:21.465535 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ce789-4031-4c80-a0ac-86b86db920fd None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:21.465818 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ce789-4031-4c80-a0ac-86b86db920fd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:21.466467 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06ce789-4031-4c80-a0ac-86b86db920fd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:21.471023 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06ce789-4031-4c80-a0ac-86b86db920fd None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:21.485152 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-d06ce789-4031-4c80-a0ac-86b86db920fd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 19:50:21.485152 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:21.485152 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:21.485152 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:21.485152 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:21.485152 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:21.485152 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:21.485152 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:21.485152 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 19:50:21.485152 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:21.485152 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:21.485152 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:21.485152 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:21.485152 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:21.485152 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:21.485152 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:21.486191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:21.486191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:21.486191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:21.486191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:21.486191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:21.486191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:21.486191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:21.486191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:21.486191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:21.486191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:21.486191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:21.486191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:21.486191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:21.486191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:21.486191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:21.486191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:21.487035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:21.487035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:21.487035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:21.487035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:21.487035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:21.487035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:21.487035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:21.487035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:21.487035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:21.487035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:21.487035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:21.487035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:21.487035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:21.487035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 19:50:21.487035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:21.487035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 349/706] 10.222.0.39 () {56 vars in 1129 bytes} [Fri Aug 5 19:50:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:21.487896 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fca2cc-5c36-42a6-8079-403011cfa12d None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:21.487896 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fca2cc-5c36-42a6-8079-403011cfa12d None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:21.487896 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fca2cc-5c36-42a6-8079-403011cfa12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:21.508233 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-f6fca2cc-5c36-42a6-8079-403011cfa12d None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:21.509471 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-f6fca2cc-5c36-42a6-8079-403011cfa12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:21.521211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:21.551075 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55525fef-2cca-4c77-b321-f51d243f9aef None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:21.594065 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d28428-1424-4a86-8fdf-9e6d46fed242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:21.594726 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d28428-1424-4a86-8fdf-9e6d46fed242 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:21.595010 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d28428-1424-4a86-8fdf-9e6d46fed242 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:21.595284 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d28428-1424-4a86-8fdf-9e6d46fed242 None admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:21.596146 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d28428-1424-4a86-8fdf-9e6d46fed242 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:21.596457 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d28428-1424-4a86-8fdf-9e6d46fed242 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:21.598181 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d28428-1424-4a86-8fdf-9e6d46fed242 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:21.613747 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 358/707] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:21.614604 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d28428-1424-4a86-8fdf-9e6d46fed242 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:21.617872 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 350/708] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:50:21.631188 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fca2cc-5c36-42a6-8079-403011cfa12d None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:21.642136 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:21.774948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d28428-1424-4a86-8fdf-9e6d46fed242 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:21.775150 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fca2cc-5c36-42a6-8079-403011cfa12d None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:21.815984 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece0c0d-257c-496d-8f1e-e8a1188dcaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:21.817023 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece0c0d-257c-496d-8f1e-e8a1188dcaf6 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:21.817511 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece0c0d-257c-496d-8f1e-e8a1188dcaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:21.818016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece0c0d-257c-496d-8f1e-e8a1188dcaf6 None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:21.818989 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dece0c0d-257c-496d-8f1e-e8a1188dcaf6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:21.819536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dece0c0d-257c-496d-8f1e-e8a1188dcaf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:21.821331 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dece0c0d-257c-496d-8f1e-e8a1188dcaf6 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:21.832950 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 359/709] 10.222.0.39 () {56 vars in 1124 bytes} [Fri Aug 5 19:50:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:21.833670 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece0c0d-257c-496d-8f1e-e8a1188dcaf6 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:21.842353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab387d2-0513-4c2b-8348-ecefeb6c7484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:21.844033 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab387d2-0513-4c2b-8348-ecefeb6c7484 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:21.844519 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab387d2-0513-4c2b-8348-ecefeb6c7484 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:21.845024 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab387d2-0513-4c2b-8348-ecefeb6c7484 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:21.846509 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab387d2-0513-4c2b-8348-ecefeb6c7484 None admin] RBAC: Adding query filter params (role.id=89919f784ad144529b07c8ab92ee7d48, scope.project.id=80379e2104644e98a2e85d4e4af1b365, user.id=95e9f085e4f04d118373b8abff27543a) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:21.847467 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab387d2-0513-4c2b-8348-ecefeb6c7484 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:21.849466 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab387d2-0513-4c2b-8348-ecefeb6c7484 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:21.857775 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 351/710] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:50:21 2022] GET /identity/v3/role_assignments?user.id=95e9f085e4f04d118373b8abff27543a&scope.project.id=80379e2104644e98a2e85d4e4af1b365&role.id=89919f784ad144529b07c8ab92ee7d48 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:21.859533 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab387d2-0513-4c2b-8348-ecefeb6c7484 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:21.876898 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:21.930483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:21.967611 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece0c0d-257c-496d-8f1e-e8a1188dcaf6 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:22.030130 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f60c7e-a9c0-4c51-8ad1-f5c8429a64a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:22.031158 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f60c7e-a9c0-4c51-8ad1-f5c8429a64a2 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:22.031671 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f60c7e-a9c0-4c51-8ad1-f5c8429a64a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:22.032149 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f60c7e-a9c0-4c51-8ad1-f5c8429a64a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:22.058177 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f60c7e-a9c0-4c51-8ad1-f5c8429a64a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:22.060475 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f60c7e-a9c0-4c51-8ad1-f5c8429a64a2 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:22.084280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-50f60c7e-a9c0-4c51-8ad1-f5c8429a64a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:22.084280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:22.084280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:22.084280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:22.084280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:22.084280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:22.084280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:22.084280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:22.084280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:22.084280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:22.084280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:22.084280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:22.084280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:22.084280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:22.084280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:22.085287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:22.085287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:22.085287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:22.085287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:22.085287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:22.085287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:22.085287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:22.085287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:22.085287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:22.085287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:22.085287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:22.085287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:22.085287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:22.085287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:22.085287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:22.085287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:22.085287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:22.086222 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:22.086222 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:22.086222 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:22.086222 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:22.086222 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 360/711] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:22.095786 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f60c7e-a9c0-4c51-8ad1-f5c8429a64a2 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:22.207646 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab387d2-0513-4c2b-8348-ecefeb6c7484 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:22.248423 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7000e1-1970-40e7-b07d-1501095355e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:22.250083 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7000e1-1970-40e7-b07d-1501095355e8 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:22.250590 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7000e1-1970-40e7-b07d-1501095355e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:22.251179 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7000e1-1970-40e7-b07d-1501095355e8 None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:22.252414 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7000e1-1970-40e7-b07d-1501095355e8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:22.252928 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7000e1-1970-40e7-b07d-1501095355e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:22.254682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7000e1-1970-40e7-b07d-1501095355e8 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:22.264094 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 352/712] 10.222.0.39 () {56 vars in 1124 bytes} [Fri Aug 5 19:50:21 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:22.318438 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f60c7e-a9c0-4c51-8ad1-f5c8429a64a2 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:22.351499 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:22.363328 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769d8a1-5255-4185-bb26-e8ebf65d9745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:22.364733 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769d8a1-5255-4185-bb26-e8ebf65d9745 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:22.365191 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769d8a1-5255-4185-bb26-e8ebf65d9745 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:22.365640 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769d8a1-5255-4185-bb26-e8ebf65d9745 None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:22.366615 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769d8a1-5255-4185-bb26-e8ebf65d9745 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:22.367124 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769d8a1-5255-4185-bb26-e8ebf65d9745 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:22.368799 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769d8a1-5255-4185-bb26-e8ebf65d9745 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:22.389960 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 361/713] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:22.397842 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7000e1-1970-40e7-b07d-1501095355e8 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:22.482259 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e2ef8-c70e-4c5e-a055-a06c0e4cce37 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:22.482640 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e2ef8-c70e-4c5e-a055-a06c0e4cce37 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:22.482808 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e2ef8-c70e-4c5e-a055-a06c0e4cce37 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:22.484480 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 362/714] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:50:22.490560 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbde97-40c1-4a5f-bd73-b14c886a65ea None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:22.491189 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbde97-40c1-4a5f-bd73-b14c886a65ea None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:22.491697 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbde97-40c1-4a5f-bd73-b14c886a65ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:22.506628 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:22.516479 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-68dbde97-40c1-4a5f-bd73-b14c886a65ea None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:22.518006 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-68dbde97-40c1-4a5f-bd73-b14c886a65ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:22.547657 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7000e1-1970-40e7-b07d-1501095355e8 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:22.592468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32f270-1cf7-4b3f-a9d5-7da57fb2d1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:22.593124 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32f270-1cf7-4b3f-a9d5-7da57fb2d1aa None admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:22.593397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32f270-1cf7-4b3f-a9d5-7da57fb2d1aa None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:22.595546 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32f270-1cf7-4b3f-a9d5-7da57fb2d1aa None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:22.596174 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff32f270-1cf7-4b3f-a9d5-7da57fb2d1aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:22.598143 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff32f270-1cf7-4b3f-a9d5-7da57fb2d1aa None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:22.608302 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 363/715] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:50:22.625151 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 353/716] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Aug 5 19:50:22 2022] POST /identity/v3/users => generated 254 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:22.637833 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dbde97-40c1-4a5f-bd73-b14c886a65ea None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:22.775401 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dbde97-40c1-4a5f-bd73-b14c886a65ea None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:22.802414 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:22.826186 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c47434-1eef-41ba-8953-e070bdef6edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:22.827195 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c47434-1eef-41ba-8953-e070bdef6edf None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:22.827728 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c47434-1eef-41ba-8953-e070bdef6edf None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:22.828163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c47434-1eef-41ba-8953-e070bdef6edf None admin] PATH_INFO: `/v3/services/network` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:22.828991 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c47434-1eef-41ba-8953-e070bdef6edf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:22.830587 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c47434-1eef-41ba-8953-e070bdef6edf None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:22.844115 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-03c47434-1eef-41ba-8953-e070bdef6edf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 19:50:22.844115 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:22.844115 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:22.844115 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:22.844115 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:22.844115 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:22.844115 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:22.844115 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:22.844115 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 19:50:22.844115 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:22.844115 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:22.844115 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:22.844115 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:22.844115 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:22.844115 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:22.844115 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:22.844115 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:22.845411 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:22.845411 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:22.845411 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:22.845411 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:22.845411 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:22.845411 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:22.845411 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:22.845411 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:22.845411 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:22.845411 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:22.845411 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:22.845411 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:22.845411 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:22.845411 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:22.845411 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:22.845411 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:22.846447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:22.846447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:22.846447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:22.846447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:22.846447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:22.846447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:22.846447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:22.846447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:22.846447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:22.846447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:22.846447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:22.846447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:22.846447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 19:50:22.846447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:22.849208 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 364/717] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:22 2022] GET /identity/v3/services/network => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:22.857086 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32f270-1cf7-4b3f-a9d5-7da57fb2d1aa None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:22.983053 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32f270-1cf7-4b3f-a9d5-7da57fb2d1aa None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:23.013567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:23.023622 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d6681-b1b7-47f2-a75b-593f92e4dbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:23.024587 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d6681-b1b7-47f2-a75b-593f92e4dbd2 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:23.024886 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d6681-b1b7-47f2-a75b-593f92e4dbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:23.025470 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d6681-b1b7-47f2-a75b-593f92e4dbd2 None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:23.026163 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d6681-b1b7-47f2-a75b-593f92e4dbd2 None admin] RBAC: Adding query filter params (name=network) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:23.026472 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d6681-b1b7-47f2-a75b-593f92e4dbd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:23.027903 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d6681-b1b7-47f2-a75b-593f92e4dbd2 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:23.033639 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 354/718] 10.222.0.39 () {56 vars in 1112 bytes} [Fri Aug 5 19:50:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:23.039241 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c47434-1eef-41ba-8953-e070bdef6edf None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:23.150201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c47434-1eef-41ba-8953-e070bdef6edf None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:23.187182 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd241cc-53f2-42e4-9b3b-04293c8eac72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:23.187816 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd241cc-53f2-42e4-9b3b-04293c8eac72 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:23.188100 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd241cc-53f2-42e4-9b3b-04293c8eac72 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:23.188379 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd241cc-53f2-42e4-9b3b-04293c8eac72 None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:23.189093 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd241cc-53f2-42e4-9b3b-04293c8eac72 None admin] RBAC: Adding query filter params (type=network) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:23.189406 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd241cc-53f2-42e4-9b3b-04293c8eac72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:23.190797 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd241cc-53f2-42e4-9b3b-04293c8eac72 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:23.198758 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 365/719] 10.222.0.39 () {56 vars in 1112 bytes} [Fri Aug 5 19:50:23 2022] GET /identity/v3/services?type=network => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:23.915072 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fc292-4fb3-4d58-bb5f-87fee4647b95 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:23.915592 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fc292-4fb3-4d58-bb5f-87fee4647b95 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:23.915704 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fc292-4fb3-4d58-bb5f-87fee4647b95 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:23.917356 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 355/720] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:23.922951 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f10aba-a6f5-440c-ab91-381b1488fbbd None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:23.923446 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f10aba-a6f5-440c-ab91-381b1488fbbd None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:23.923521 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f10aba-a6f5-440c-ab91-381b1488fbbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:23.958209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-63f10aba-a6f5-440c-ab91-381b1488fbbd None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:23.959403 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-63f10aba-a6f5-440c-ab91-381b1488fbbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:24.066835 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:24.272945 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 366/721] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:50:24.310166 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fc292-4fb3-4d58-bb5f-87fee4647b95 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:24.461328 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fc292-4fb3-4d58-bb5f-87fee4647b95 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:24.509210 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3d7a9-85a7-4354-95ad-c5d00750108d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:24.509859 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3d7a9-85a7-4354-95ad-c5d00750108d None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:24.510143 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3d7a9-85a7-4354-95ad-c5d00750108d None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:24.510417 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3d7a9-85a7-4354-95ad-c5d00750108d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:24.529847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac3d7a9-85a7-4354-95ad-c5d00750108d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:24.534711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-2ac3d7a9-85a7-4354-95ad-c5d00750108d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:50:24.534711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:24.534711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:24.534711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:24.534711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:24.534711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:24.534711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:24.534711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:24.534711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:24.534711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:24.534711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:24.534711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:24.534711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:24.534711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:24.534711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:24.534711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:24.535627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:24.535627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:24.535627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:24.535627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:24.535627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:24.535627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:24.535627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:24.535627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:24.535627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:24.535627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:24.535627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:24.535627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:24.535627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:24.535627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:24.535627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:24.535627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:24.536904 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:24.536904 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:24.536904 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:24.536904 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:24.536904 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:24.536904 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:24.536904 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:24.536904 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:24.536904 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:24.536904 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:24.536904 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:50:24.536904 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:24.536904 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 356/722] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:50:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:24.544306 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f10aba-a6f5-440c-ab91-381b1488fbbd None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:24.615582 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e01a47-cfa4-4f01-bf79-da58751311f8 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:24.615925 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e01a47-cfa4-4f01-bf79-da58751311f8 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:24.616134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e01a47-cfa4-4f01-bf79-da58751311f8 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:24.617501 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 357/723] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:50:24.623931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30acab-66e3-43e0-8ee1-0189c7e14621 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:24.624239 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30acab-66e3-43e0-8ee1-0189c7e14621 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:24.624513 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30acab-66e3-43e0-8ee1-0189c7e14621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:24.657636 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-4b30acab-66e3-43e0-8ee1-0189c7e14621 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:24.658854 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-4b30acab-66e3-43e0-8ee1-0189c7e14621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:24.681703 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f10aba-a6f5-440c-ab91-381b1488fbbd None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:24.729606 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd01cbb-8a92-46bd-931c-f64e0984fb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:24.730351 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd01cbb-8a92-46bd-931c-f64e0984fb3c None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:24.731174 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd01cbb-8a92-46bd-931c-f64e0984fb3c None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:24.731451 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd01cbb-8a92-46bd-931c-f64e0984fb3c None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:24.732191 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd01cbb-8a92-46bd-931c-f64e0984fb3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:24.732501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd01cbb-8a92-46bd-931c-f64e0984fb3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:24.734031 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd01cbb-8a92-46bd-931c-f64e0984fb3c None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:24.743294 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 367/724] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:50:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:24.752373 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:24.753684 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd01cbb-8a92-46bd-931c-f64e0984fb3c None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:24.790356 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 358/725] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:50:24.814701 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:24.833163 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30acab-66e3-43e0-8ee1-0189c7e14621 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:24.941878 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd01cbb-8a92-46bd-931c-f64e0984fb3c None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:24.994792 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30acab-66e3-43e0-8ee1-0189c7e14621 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:25.020921 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8baef9-19b5-4c72-aefa-d6428b079d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:25.021982 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8baef9-19b5-4c72-aefa-d6428b079d3a None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:25.022651 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8baef9-19b5-4c72-aefa-d6428b079d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:25.023253 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8baef9-19b5-4c72-aefa-d6428b079d3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:25.035805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae259a1-9d8d-406e-8a22-2230fa55ae7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:25.036452 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae259a1-9d8d-406e-8a22-2230fa55ae7b None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:25.036759 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae259a1-9d8d-406e-8a22-2230fa55ae7b None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:25.037029 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae259a1-9d8d-406e-8a22-2230fa55ae7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:25.049767 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae259a1-9d8d-406e-8a22-2230fa55ae7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:25.051087 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8baef9-19b5-4c72-aefa-d6428b079d3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:25.052741 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8baef9-19b5-4c72-aefa-d6428b079d3a None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:25.054841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-dae259a1-9d8d-406e-8a22-2230fa55ae7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:50:25.054841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:25.054841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:25.054841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:25.054841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:25.054841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:25.054841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:25.054841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:25.054841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:25.054841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:25.054841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:25.054841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:25.054841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:25.054841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:25.054841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:25.054841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:25.055972 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:25.055972 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:25.055972 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:25.055972 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:25.055972 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:25.055972 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:25.055972 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:25.055972 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:25.055972 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:25.055972 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:25.055972 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:25.055972 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:25.055972 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:25.055972 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:25.055972 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:25.055972 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:25.057052 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:25.057052 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:25.057052 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:25.057052 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:25.057052 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:25.057052 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:25.057052 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:25.057052 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:25.057052 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:25.057052 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:25.057052 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:50:25.057052 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:25.057052 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 359/726] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:50:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:25.065645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae259a1-9d8d-406e-8a22-2230fa55ae7b None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:25.089350 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-4e8baef9-19b5-4c72-aefa-d6428b079d3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:25.089350 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:25.089350 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:25.089350 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:25.089350 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:25.089350 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:25.089350 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:25.089350 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:25.089350 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:25.089350 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:25.089350 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:25.089350 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:25.089350 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:25.089350 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:25.089350 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:25.090581 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:25.090581 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:25.090581 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:25.090581 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:25.090581 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:25.090581 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:25.090581 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:25.090581 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:25.090581 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:25.090581 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:25.090581 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:25.090581 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:25.090581 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:25.090581 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:25.090581 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:25.090581 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:25.090581 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:25.091499 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:25.091499 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:25.091499 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:25.091499 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:25.091499 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 368/727] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:25.098414 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8baef9-19b5-4c72-aefa-d6428b079d3a None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:25.137208 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:25.202343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:25.211151 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae259a1-9d8d-406e-8a22-2230fa55ae7b None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:25.216228 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8baef9-19b5-4c72-aefa-d6428b079d3a None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:25.251227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edf6505-132b-4954-9d44-bd0ece2fc0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:25.252278 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf6505-132b-4954-9d44-bd0ece2fc0fb None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:25.252472 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64805fbc-0135-463d-83a3-c7f64b80e60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:25.252989 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf6505-132b-4954-9d44-bd0ece2fc0fb None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:25.253095 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64805fbc-0135-463d-83a3-c7f64b80e60b None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:25.253330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64805fbc-0135-463d-83a3-c7f64b80e60b None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:25.253599 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64805fbc-0135-463d-83a3-c7f64b80e60b None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:25.253977 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf6505-132b-4954-9d44-bd0ece2fc0fb None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:25.254286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64805fbc-0135-463d-83a3-c7f64b80e60b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:25.254586 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64805fbc-0135-463d-83a3-c7f64b80e60b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:25.255296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edf6505-132b-4954-9d44-bd0ece2fc0fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:25.255797 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edf6505-132b-4954-9d44-bd0ece2fc0fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:25.256001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64805fbc-0135-463d-83a3-c7f64b80e60b None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:25.257712 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edf6505-132b-4954-9d44-bd0ece2fc0fb None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:25.271962 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 369/729] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:25.271962 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51fc8b-6cf3-4e92-b5ba-1c7ec7921c4f None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:25.271962 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51fc8b-6cf3-4e92-b5ba-1c7ec7921c4f None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:25.271962 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51fc8b-6cf3-4e92-b5ba-1c7ec7921c4f None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:25.272591 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 360/730] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:50:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:25.274191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edf6505-132b-4954-9d44-bd0ece2fc0fb None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:25.276338 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 370/730] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:25.277359 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51fc8b-6cf3-4e92-b5ba-1c7ec7921c4f None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:25.408102 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51fc8b-6cf3-4e92-b5ba-1c7ec7921c4f None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:25.418625 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edf6505-132b-4954-9d44-bd0ece2fc0fb None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:25.422088 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:25.446711 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c7d5a7-61c0-43c6-9632-a72527d6b9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:25.447356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7d5a7-61c0-43c6-9632-a72527d6b9ae None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:25.447633 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7d5a7-61c0-43c6-9632-a72527d6b9ae None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:25.447902 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7d5a7-61c0-43c6-9632-a72527d6b9ae None admin] PATH_INFO: `/v3/users/swift` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:25.458063 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c7d5a7-61c0-43c6-9632-a72527d6b9ae None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:25.459620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c7d5a7-61c0-43c6-9632-a72527d6b9ae None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:25.468230 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffca4f4-3be8-49a9-b032-78ff10610374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:25.468899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffca4f4-3be8-49a9-b032-78ff10610374 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:25.469232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffca4f4-3be8-49a9-b032-78ff10610374 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:25.469516 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffca4f4-3be8-49a9-b032-78ff10610374 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:25.472600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-49c7d5a7-61c0-43c6-9632-a72527d6b9ae None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:50:25.472600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:25.472600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:25.472600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:25.472600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:25.472600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:25.472600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:25.472600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:25.472600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:25.472600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:25.472600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:25.472600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:25.472600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:25.472600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:25.472600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:25.472600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:25.473536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:25.473536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:25.473536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:25.473536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:25.473536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:25.473536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:25.473536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:25.473536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:25.473536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:25.473536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:25.473536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:25.473536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:25.473536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:25.473536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:25.473536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:25.473536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:25.473536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:25.473536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:25.474474 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:25.474474 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:25.474474 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:25.474474 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:25.474474 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:25.474474 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:25.474474 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:25.474474 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:25.474474 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:50:25.474474 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:25.474474 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 371/731] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:50:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:25.475548 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8032aa-e86c-4843-91d4-dca5e5507c7e None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:25.475706 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8032aa-e86c-4843-91d4-dca5e5507c7e None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:25.477581 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8032aa-e86c-4843-91d4-dca5e5507c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:25.488055 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffca4f4-3be8-49a9-b032-78ff10610374 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:25.489527 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffca4f4-3be8-49a9-b032-78ff10610374 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:25.500051 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-0c8032aa-e86c-4843-91d4-dca5e5507c7e None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:25.501455 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-0c8032aa-e86c-4843-91d4-dca5e5507c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:25.503894 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:25.509423 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-5ffca4f4-3be8-49a9-b032-78ff10610374 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:25.509423 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:25.509423 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:25.509423 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:25.509423 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:25.509423 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:25.509423 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:25.509423 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:25.509423 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:25.509423 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:25.509423 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:25.509423 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:25.509423 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:25.509423 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:25.509423 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:25.510506 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:25.510506 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:25.510506 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:25.510506 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:25.510506 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:25.510506 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:25.510506 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:25.510506 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:25.510506 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:25.510506 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:25.510506 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:25.510506 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:25.510506 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:25.510506 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:25.510506 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:25.510506 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:25.510506 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:25.512241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:25.512241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:25.512241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:25.512241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:25.515216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 361/732] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:25.516709 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffca4f4-3be8-49a9-b032-78ff10610374 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:25.599010 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 372/733] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:50:25.600429 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8032aa-e86c-4843-91d4-dca5e5507c7e None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:25.662683 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffca4f4-3be8-49a9-b032-78ff10610374 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:25.706890 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94584781-cb5a-40d7-b183-232c4fa6f04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:25.708566 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94584781-cb5a-40d7-b183-232c4fa6f04d None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:25.709257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94584781-cb5a-40d7-b183-232c4fa6f04d None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:25.709849 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94584781-cb5a-40d7-b183-232c4fa6f04d None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:25.711590 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94584781-cb5a-40d7-b183-232c4fa6f04d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:25.712080 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94584781-cb5a-40d7-b183-232c4fa6f04d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:25.714189 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94584781-cb5a-40d7-b183-232c4fa6f04d None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:25.733055 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8032aa-e86c-4843-91d4-dca5e5507c7e None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:25.752696 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 362/734] 10.222.0.39 () {56 vars in 1135 bytes} [Fri Aug 5 19:50:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:25.752696 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94584781-cb5a-40d7-b183-232c4fa6f04d None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:25.766927 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:25.782036 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66195871-8c2e-472e-84e9-cb4ab30ef1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:25.783110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66195871-8c2e-472e-84e9-cb4ab30ef1c8 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:25.783604 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66195871-8c2e-472e-84e9-cb4ab30ef1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:25.784105 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66195871-8c2e-472e-84e9-cb4ab30ef1c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:25.785436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66195871-8c2e-472e-84e9-cb4ab30ef1c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:25.785988 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66195871-8c2e-472e-84e9-cb4ab30ef1c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:25.787793 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66195871-8c2e-472e-84e9-cb4ab30ef1c8 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:25.796903 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:25.802190 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 373/735] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:25.804886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df4aad-fb12-46bf-8ccd-ebb4293dd9b7 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:25.805336 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df4aad-fb12-46bf-8ccd-ebb4293dd9b7 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:25.805722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df4aad-fb12-46bf-8ccd-ebb4293dd9b7 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:25.808028 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 374/736] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:25.809308 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df4aad-fb12-46bf-8ccd-ebb4293dd9b7 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:25.871841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94584781-cb5a-40d7-b183-232c4fa6f04d None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:25.911888 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be579ca7-35d1-4a80-9591-ed7ec27917ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:25.912921 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be579ca7-35d1-4a80-9591-ed7ec27917ba None admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:25.913141 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be579ca7-35d1-4a80-9591-ed7ec27917ba None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:25.913412 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be579ca7-35d1-4a80-9591-ed7ec27917ba None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:25.914107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be579ca7-35d1-4a80-9591-ed7ec27917ba None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:25.915459 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be579ca7-35d1-4a80-9591-ed7ec27917ba None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:25.931539 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df4aad-fb12-46bf-8ccd-ebb4293dd9b7 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:25.933144 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 363/737] 10.222.0.39 () {60 vars in 1141 bytes} [Fri Aug 5 19:50:25 2022] POST /identity/v3/services => generated 251 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:25.934831 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be579ca7-35d1-4a80-9591-ed7ec27917ba None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:25.989453 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b7998-c891-462d-a2d0-1ce65ffd9293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:25.990708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b7998-c891-462d-a2d0-1ce65ffd9293 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:25.992447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b7998-c891-462d-a2d0-1ce65ffd9293 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:25.992798 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b7998-c891-462d-a2d0-1ce65ffd9293 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:26.053233 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:26.081837 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:26.091453 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b7998-c891-462d-a2d0-1ce65ffd9293 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:26.092050 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be579ca7-35d1-4a80-9591-ed7ec27917ba None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:26.093034 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b7998-c891-462d-a2d0-1ce65ffd9293 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:26.145556 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0997c3-dd1b-40ba-961a-c18089856123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:26.146114 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0997c3-dd1b-40ba-961a-c18089856123 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:26.146390 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0997c3-dd1b-40ba-961a-c18089856123 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:26.146665 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0997c3-dd1b-40ba-961a-c18089856123 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:26.156730 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0997c3-dd1b-40ba-961a-c18089856123 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:26.158175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0997c3-dd1b-40ba-961a-c18089856123 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:26.171618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-8c0997c3-dd1b-40ba-961a-c18089856123 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:50:26.171618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:26.171618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:26.171618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:26.171618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:26.171618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:26.171618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:26.171618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:26.171618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:26.171618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:26.171618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:26.171618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:26.171618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:26.171618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:26.171618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:26.171618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:26.172652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:26.172652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:26.172652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:26.172652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:26.172652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:26.172652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:26.172652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:26.172652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:26.172652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:26.172652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:26.172652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:26.172652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:26.172652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:26.172652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:26.172652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:26.172652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:26.172652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:26.172652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:26.173728 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:26.173728 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:26.173728 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:26.173728 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:26.173728 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:26.173728 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:26.173728 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:26.173728 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:26.173728 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:50:26.173728 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:26.173728 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 364/738] 10.222.0.39 () {56 vars in 1093 bytes} [Fri Aug 5 19:50:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:26.174307 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e7c15-cd57-44a1-b446-2e65dc0092aa None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:26.174665 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e7c15-cd57-44a1-b446-2e65dc0092aa None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:26.174759 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e7c15-cd57-44a1-b446-2e65dc0092aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:26.197552 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-d78e7c15-cd57-44a1-b446-2e65dc0092aa None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:26.198732 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-d78e7c15-cd57-44a1-b446-2e65dc0092aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:26.220134 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 375/739] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:50:25 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:50:26.225102 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63eb28-3b5d-4f1a-9bcf-6a3b659660c9 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:26.225102 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63eb28-3b5d-4f1a-9bcf-6a3b659660c9 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:26.225102 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63eb28-3b5d-4f1a-9bcf-6a3b659660c9 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:26.225102 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 376/740] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:26.225102 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a63eb28-3b5d-4f1a-9bcf-6a3b659660c9 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:26.279286 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 365/741] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:26 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:50:26.280497 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e7c15-cd57-44a1-b446-2e65dc0092aa None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:26.357772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:26.366546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a63eb28-3b5d-4f1a-9bcf-6a3b659660c9 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:26.372473 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:26.407651 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5eb27e-ea26-4069-a750-d8ef7a7c1efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:26.408271 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5eb27e-ea26-4069-a750-d8ef7a7c1efa None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:26.408548 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5eb27e-ea26-4069-a750-d8ef7a7c1efa None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:26.408844 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5eb27e-ea26-4069-a750-d8ef7a7c1efa None admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:26.409640 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5eb27e-ea26-4069-a750-d8ef7a7c1efa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:26.409942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5eb27e-ea26-4069-a750-d8ef7a7c1efa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:26.411358 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5eb27e-ea26-4069-a750-d8ef7a7c1efa None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:26.419163 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e7c15-cd57-44a1-b446-2e65dc0092aa None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:26.431869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 377/742] 10.222.0.39 () {56 vars in 1133 bytes} [Fri Aug 5 19:50:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:26.433031 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba05400-d557-4809-a568-7f174845a20e None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:26.433839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba05400-d557-4809-a568-7f174845a20e None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:26.434447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba05400-d557-4809-a568-7f174845a20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:26.453305 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-0ba05400-d557-4809-a568-7f174845a20e None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:26.454489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-0ba05400-d557-4809-a568-7f174845a20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:26.467505 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb75e11-3e62-4bdd-857b-6068597c9c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:26.468144 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb75e11-3e62-4bdd-857b-6068597c9c13 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:26.468419 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb75e11-3e62-4bdd-857b-6068597c9c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:26.468718 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb75e11-3e62-4bdd-857b-6068597c9c13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:26.479556 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb75e11-3e62-4bdd-857b-6068597c9c13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:26.481702 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb75e11-3e62-4bdd-857b-6068597c9c13 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:26.498293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-bfb75e11-3e62-4bdd-857b-6068597c9c13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:26.498293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:26.498293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:26.498293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:26.498293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:26.498293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:26.498293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:26.498293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:26.498293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:26.498293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:26.498293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:26.498293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:26.498293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:26.498293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:26.498293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:26.499241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:26.499241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:26.499241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:26.499241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:26.499241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:26.499241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:26.499241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:26.499241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:26.499241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:26.499241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:26.499241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:26.499241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:26.499241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:26.499241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:26.499241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:26.499241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:26.499241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:26.500844 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:26.500844 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:26.500844 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:26.500844 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:26.500844 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 366/743] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:26.500844 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb75e11-3e62-4bdd-857b-6068597c9c13 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:26.531297 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 378/744] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:26 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:50:26.532556 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba05400-d557-4809-a568-7f174845a20e None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:26.619132 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb75e11-3e62-4bdd-857b-6068597c9c13 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:26.636368 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:26.638859 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:26.657772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba05400-d557-4809-a568-7f174845a20e None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:26.663094 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975cd68-4014-4865-96f2-bf3d13b82c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:26.663356 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975cd68-4014-4865-96f2-bf3d13b82c1f None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:26.663570 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975cd68-4014-4865-96f2-bf3d13b82c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:26.663843 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975cd68-4014-4865-96f2-bf3d13b82c1f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:26.674498 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975cd68-4014-4865-96f2-bf3d13b82c1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:26.679832 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-2975cd68-4014-4865-96f2-bf3d13b82c1f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:26.679832 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:26.679832 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:26.679832 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:26.679832 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:26.679832 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:26.679832 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:26.679832 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:26.679832 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:26.679832 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:26.679832 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:26.679832 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:26.679832 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:26.679832 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:26.679832 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:26.679832 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:26.680813 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:26.680813 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:26.680813 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:26.680813 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:26.680813 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:26.680813 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:26.680813 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:26.680813 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:26.680813 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:26.680813 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:26.680813 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:26.680813 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:26.680813 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:26.680813 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:26.680813 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:26.680813 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:26.681645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:26.681645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:26.681645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:26.681645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:26.681645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:26.681645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:26.681645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:26.681645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:26.681645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:26.681645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:26.681645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:26.681645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:26.681645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 367/745] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Aug 5 19:50:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:26.682397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975cd68-4014-4865-96f2-bf3d13b82c1f None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:26.702518 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d501d8-01be-431e-a293-0d6409ba256f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:26.703436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d501d8-01be-431e-a293-0d6409ba256f None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:26.703436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d501d8-01be-431e-a293-0d6409ba256f None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:26.703807 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d501d8-01be-431e-a293-0d6409ba256f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:26.715000 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d501d8-01be-431e-a293-0d6409ba256f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:26.716467 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d501d8-01be-431e-a293-0d6409ba256f None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:26.745057 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-c3d501d8-01be-431e-a293-0d6409ba256f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:26.745057 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:26.745057 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:26.745057 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:26.745057 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:26.745057 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:26.745057 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:26.745057 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:26.745057 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:26.745057 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:26.745057 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:26.745057 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:26.745057 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:26.745057 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:26.745057 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:26.748325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:26.748325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:26.748325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:26.748325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:26.748325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:26.748325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:26.748325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:26.748325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:26.748325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:26.748325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:26.748325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:26.748325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:26.748325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:26.748325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:26.748325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:26.748325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:26.748325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:26.749250 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:26.749250 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:26.749250 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:26.749250 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:26.749250 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 379/746] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:26.749250 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d501d8-01be-431e-a293-0d6409ba256f None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:26.816858 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975cd68-4014-4865-96f2-bf3d13b82c1f None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:26.857977 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a988639-5e7e-4a8b-af8b-1adfed6ed8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:26.858714 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a988639-5e7e-4a8b-af8b-1adfed6ed8fe None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:26.859123 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a988639-5e7e-4a8b-af8b-1adfed6ed8fe None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:26.859512 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a988639-5e7e-4a8b-af8b-1adfed6ed8fe None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:26.860286 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a988639-5e7e-4a8b-af8b-1adfed6ed8fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:26.860609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a988639-5e7e-4a8b-af8b-1adfed6ed8fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:26.862090 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a988639-5e7e-4a8b-af8b-1adfed6ed8fe None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:26.865490 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d501d8-01be-431e-a293-0d6409ba256f None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:26.881885 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 368/747] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:26.881885 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a988639-5e7e-4a8b-af8b-1adfed6ed8fe None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:26.913479 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e0982-76a7-4f78-8b54-970e8638099d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:26.914221 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e0982-76a7-4f78-8b54-970e8638099d None admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:26.914509 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e0982-76a7-4f78-8b54-970e8638099d None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:26.914785 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e0982-76a7-4f78-8b54-970e8638099d None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:26.915484 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e0982-76a7-4f78-8b54-970e8638099d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:26.916877 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e0982-76a7-4f78-8b54-970e8638099d None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:26.920540 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:26.921109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:26.936220 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 380/748] 10.222.0.39 () {60 vars in 1141 bytes} [Fri Aug 5 19:50:26 2022] POST /identity/v3/services => generated 240 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:26.937163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e0982-76a7-4f78-8b54-970e8638099d None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:27.015780 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a988639-5e7e-4a8b-af8b-1adfed6ed8fe None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:27.071881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e0982-76a7-4f78-8b54-970e8638099d None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:27.075439 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034641d-ad28-44b6-88b2-f1aca1c3008f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:27.076869 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034641d-ad28-44b6-88b2-f1aca1c3008f None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:27.077372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034641d-ad28-44b6-88b2-f1aca1c3008f None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:27.077854 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034641d-ad28-44b6-88b2-f1aca1c3008f None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:27.078806 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f034641d-ad28-44b6-88b2-f1aca1c3008f None admin] RBAC: Adding query filter params (name=service) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:27.079319 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f034641d-ad28-44b6-88b2-f1aca1c3008f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:27.081109 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f034641d-ad28-44b6-88b2-f1aca1c3008f None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:27.096675 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 369/749] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Aug 5 19:50:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:27.097976 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034641d-ad28-44b6-88b2-f1aca1c3008f None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:27.115706 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d9b0d-588c-48ba-ad22-5170f49416fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:27.116524 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d9b0d-588c-48ba-ad22-5170f49416fd None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:27.116874 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d9b0d-588c-48ba-ad22-5170f49416fd None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:27.117153 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d9b0d-588c-48ba-ad22-5170f49416fd None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:27.117924 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d9b0d-588c-48ba-ad22-5170f49416fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:27.118239 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d9b0d-588c-48ba-ad22-5170f49416fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:27.119630 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d9b0d-588c-48ba-ad22-5170f49416fd None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:27.138399 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 381/750] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:27.139587 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d9b0d-588c-48ba-ad22-5170f49416fd None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:27.215272 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:27.226481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:27.235745 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034641d-ad28-44b6-88b2-f1aca1c3008f None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:27.275432 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f82dc4-be14-43b9-a530-f15c46c1657a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:27.276080 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f82dc4-be14-43b9-a530-f15c46c1657a None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:27.276364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f82dc4-be14-43b9-a530-f15c46c1657a None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:27.276682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f82dc4-be14-43b9-a530-f15c46c1657a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:27.276809 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d9b0d-588c-48ba-ad22-5170f49416fd None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:27.284471 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f82dc4-be14-43b9-a530-f15c46c1657a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:27.286440 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f82dc4-be14-43b9-a530-f15c46c1657a None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:27.300912 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-82f82dc4-be14-43b9-a530-f15c46c1657a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:27.300912 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:27.300912 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:27.300912 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:27.300912 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:27.300912 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:27.300912 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:27.300912 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:27.300912 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:27.300912 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:27.300912 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:27.300912 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:27.300912 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:27.300912 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:27.300912 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:27.302004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:27.302004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:27.302004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:27.302004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:27.302004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:27.302004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:27.302004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:27.302004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:27.302004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:27.302004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:27.302004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:27.302004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:27.302004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:27.302004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:27.302004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:27.302004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:27.302004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:27.302948 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:27.302948 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:27.302948 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:27.302948 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:27.302948 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:27.302948 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:27.302948 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:27.302948 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:27.306972 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 370/751] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:27.308009 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f82dc4-be14-43b9-a530-f15c46c1657a None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:27.321868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54128e68-9ca2-41d1-a8ed-da009432162b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:27.322897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54128e68-9ca2-41d1-a8ed-da009432162b None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:27.323326 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54128e68-9ca2-41d1-a8ed-da009432162b None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:27.323757 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54128e68-9ca2-41d1-a8ed-da009432162b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:27.336974 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54128e68-9ca2-41d1-a8ed-da009432162b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:27.338937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54128e68-9ca2-41d1-a8ed-da009432162b None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:27.356831 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-54128e68-9ca2-41d1-a8ed-da009432162b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:27.356831 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:27.356831 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:27.356831 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:27.356831 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:27.356831 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:27.356831 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:27.356831 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:27.356831 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:27.356831 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:27.356831 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:27.356831 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:27.356831 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:27.356831 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:27.356831 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:27.357997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:27.357997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:27.357997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:27.357997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:27.357997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:27.357997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:27.357997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:27.357997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:27.357997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:27.357997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:27.357997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:27.357997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:27.357997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:27.357997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:27.357997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:27.357997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:27.357997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:27.359059 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:27.359059 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:27.359059 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:27.359059 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:27.360796 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 382/752] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:27.361786 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54128e68-9ca2-41d1-a8ed-da009432162b None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:27.455116 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f82dc4-be14-43b9-a530-f15c46c1657a None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:27.477600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54128e68-9ca2-41d1-a8ed-da009432162b None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:27.479489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:27.498192 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb3bea-0c69-466d-ba5f-5a51c2c611dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:27.498852 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb3bea-0c69-466d-ba5f-5a51c2c611dd None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:27.499134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb3bea-0c69-466d-ba5f-5a51c2c611dd None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:27.499439 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb3bea-0c69-466d-ba5f-5a51c2c611dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:27.507222 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb3bea-0c69-466d-ba5f-5a51c2c611dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:27.508709 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb3bea-0c69-466d-ba5f-5a51c2c611dd None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:27.519269 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3dde38-00d5-4412-ad34-dd15cedb1dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:27.520172 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3dde38-00d5-4412-ad34-dd15cedb1dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:27.520625 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3dde38-00d5-4412-ad34-dd15cedb1dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:27.521165 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3dde38-00d5-4412-ad34-dd15cedb1dc4 None admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:27.522247 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3dde38-00d5-4412-ad34-dd15cedb1dc4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:27.522710 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3dde38-00d5-4412-ad34-dd15cedb1dc4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:27.522873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-6edb3bea-0c69-466d-ba5f-5a51c2c611dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:27.522873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:27.522873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:27.522873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:27.522873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:27.522873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:27.522873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:27.522873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:27.522873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:27.522873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:27.522873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:27.522873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:27.522873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:27.522873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:27.522873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:27.524185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:27.524185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:27.524185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:27.524185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:27.524185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:27.524185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:27.524185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:27.524185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:27.524185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:27.524185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:27.524185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:27.524185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:27.524185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:27.524185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:27.524185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:27.524185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:27.524185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:27.525202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:27.525202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:27.525202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:27.525202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:27.525202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:27.525202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:27.525202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:27.525202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:27.525202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 371/753] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:27.525708 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb3bea-0c69-466d-ba5f-5a51c2c611dd None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:27.528068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3dde38-00d5-4412-ad34-dd15cedb1dc4 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:27.543068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 383/754] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:27.545111 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:27.545977 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3dde38-00d5-4412-ad34-dd15cedb1dc4 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:27.664369 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb3bea-0c69-466d-ba5f-5a51c2c611dd None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:27.680496 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3dde38-00d5-4412-ad34-dd15cedb1dc4 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:27.712740 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b32579-c1ba-450d-9739-5ba9741a6449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:27.713290 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b32579-c1ba-450d-9739-5ba9741a6449 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:27.713573 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b32579-c1ba-450d-9739-5ba9741a6449 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:27.713849 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b32579-c1ba-450d-9739-5ba9741a6449 None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:27.715215 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b32579-c1ba-450d-9739-5ba9741a6449 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:27.715675 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b32579-c1ba-450d-9739-5ba9741a6449 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:27.717440 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b32579-c1ba-450d-9739-5ba9741a6449 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:27.730421 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d312597-e683-4940-8505-1b4d38e373c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:27.731275 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d312597-e683-4940-8505-1b4d38e373c8 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:27.731696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d312597-e683-4940-8505-1b4d38e373c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:27.732108 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d312597-e683-4940-8505-1b4d38e373c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:27.732726 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 372/755] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:27.733221 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d312597-e683-4940-8505-1b4d38e373c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:27.733586 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b32579-c1ba-450d-9739-5ba9741a6449 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:27.733824 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d312597-e683-4940-8505-1b4d38e373c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:27.735482 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d312597-e683-4940-8505-1b4d38e373c8 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:27.762991 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 384/756] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:27.764556 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d312597-e683-4940-8505-1b4d38e373c8 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:27.800003 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:27.857449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:27.891886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d312597-e683-4940-8505-1b4d38e373c8 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:27.899299 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b32579-c1ba-450d-9739-5ba9741a6449 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:27.932010 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659eb3a0-00f0-40c2-bea9-cc0be8613937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:27.932684 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659eb3a0-00f0-40c2-bea9-cc0be8613937 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:27.933841 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659eb3a0-00f0-40c2-bea9-cc0be8613937 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:27.933841 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659eb3a0-00f0-40c2-bea9-cc0be8613937 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:27.942138 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659eb3a0-00f0-40c2-bea9-cc0be8613937 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:27.943727 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659eb3a0-00f0-40c2-bea9-cc0be8613937 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:27.951264 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039570b2-4a76-45f4-8b31-cbcbbb0f7ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:27.952070 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039570b2-4a76-45f4-8b31-cbcbbb0f7ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:27.952487 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039570b2-4a76-45f4-8b31-cbcbbb0f7ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:27.952919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039570b2-4a76-45f4-8b31-cbcbbb0f7ab8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:27.953843 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039570b2-4a76-45f4-8b31-cbcbbb0f7ab8 None admin] RBAC: Adding query filter params (role.id=89919f784ad144529b07c8ab92ee7d48, scope.project.id=80379e2104644e98a2e85d4e4af1b365, user.id=91d3d43cb0bd4ca3b8e8f10e29cc421c) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:27.954314 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039570b2-4a76-45f4-8b31-cbcbbb0f7ab8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:27.955784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-659eb3a0-00f0-40c2-bea9-cc0be8613937 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:50:27.955784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:27.955784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:27.955784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:27.955784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:27.955784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:27.955784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:27.955784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:27.955784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:27.955784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:27.955784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:27.955784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:27.955784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:27.955784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:27.955784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:27.955784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:27.956906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:27.956906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:27.956906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:27.956906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:27.956906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:27.956906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:27.956906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:27.956906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:27.956906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:27.956906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:27.956906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:27.956906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:27.956906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:27.956906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:27.956906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:27.956906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:27.956906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:27.956906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:27.957902 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:27.957902 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:27.957902 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:27.957902 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:27.957902 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:27.957902 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:27.957902 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:27.957902 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:27.957902 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:50:27.957902 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:27.957902 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 385/757] 10.222.0.39 () {56 vars in 1117 bytes} [Fri Aug 5 19:50:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:27.958487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659eb3a0-00f0-40c2-bea9-cc0be8613937 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:27.959001 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039570b2-4a76-45f4-8b31-cbcbbb0f7ab8 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:27.967671 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 373/758] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:50:27 2022] GET /identity/v3/role_assignments?user.id=91d3d43cb0bd4ca3b8e8f10e29cc421c&scope.project.id=80379e2104644e98a2e85d4e4af1b365&role.id=89919f784ad144529b07c8ab92ee7d48 => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:27.968870 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039570b2-4a76-45f4-8b31-cbcbbb0f7ab8 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:28.099518 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039570b2-4a76-45f4-8b31-cbcbbb0f7ab8 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:28.150329 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:28.157413 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709a634c-a879-44b3-8380-96735eaf257c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:28.158688 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a634c-a879-44b3-8380-96735eaf257c None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:28.159164 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a634c-a879-44b3-8380-96735eaf257c None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:28.159439 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a634c-a879-44b3-8380-96735eaf257c None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:28.160265 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709a634c-a879-44b3-8380-96735eaf257c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:28.160683 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709a634c-a879-44b3-8380-96735eaf257c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:28.162146 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709a634c-a879-44b3-8380-96735eaf257c None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:28.176348 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:28.184104 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659eb3a0-00f0-40c2-bea9-cc0be8613937 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:28.192185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 374/759] 10.222.0.39 () {56 vars in 1149 bytes} [Fri Aug 5 19:50:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:28.193568 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd05fe-2a4d-4973-b603-19a6b5963dca None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:28.193851 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd05fe-2a4d-4973-b603-19a6b5963dca None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:28.194123 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd05fe-2a4d-4973-b603-19a6b5963dca None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:28.200908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 375/760] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:28.202531 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd05fe-2a4d-4973-b603-19a6b5963dca None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:28.235473 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e85b3f3-7a03-4fb3-8e02-cf1ec96367c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:28.236147 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85b3f3-7a03-4fb3-8e02-cf1ec96367c5 None admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:28.236423 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85b3f3-7a03-4fb3-8e02-cf1ec96367c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:28.236722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85b3f3-7a03-4fb3-8e02-cf1ec96367c5 None admin] PATH_INFO: `/v3/projects/80379e2104644e98a2e85d4e4af1b365/users/95e9f085e4f04d118373b8abff27543a/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:28.239664 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e85b3f3-7a03-4fb3-8e02-cf1ec96367c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=80379e2104644e98a2e85d4e4af1b365, user_id=95e9f085e4f04d118373b8abff27543a, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:28.241181 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e85b3f3-7a03-4fb3-8e02-cf1ec96367c5 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:28.264618 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 386/761] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Aug 5 19:50:27 2022] PUT /identity/v3/projects/80379e2104644e98a2e85d4e4af1b365/users/95e9f085e4f04d118373b8abff27543a/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:28.264618 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812d394-36da-4e77-aa2b-175391014ac8 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:28.264618 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812d394-36da-4e77-aa2b-175391014ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:28.264618 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812d394-36da-4e77-aa2b-175391014ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:28.287190 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-1812d394-36da-4e77-aa2b-175391014ac8 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:28.288751 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-1812d394-36da-4e77-aa2b-175391014ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:28.382462 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd05fe-2a4d-4973-b603-19a6b5963dca None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:28.387808 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 387/762] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:28.412441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812d394-36da-4e77-aa2b-175391014ac8 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:28.422430 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5227bad2-2071-4515-afe9-220db895b2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:28.423146 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227bad2-2071-4515-afe9-220db895b2cb None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:28.423496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227bad2-2071-4515-afe9-220db895b2cb None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:28.423844 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227bad2-2071-4515-afe9-220db895b2cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:28.492081 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:28.495130 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:28.500072 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5227bad2-2071-4515-afe9-220db895b2cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:28.502011 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5227bad2-2071-4515-afe9-220db895b2cb None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:28.531148 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812d394-36da-4e77-aa2b-175391014ac8 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:28.573356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d072e52-69e9-4f25-b4df-c38f94858727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:28.574512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d072e52-69e9-4f25-b4df-c38f94858727 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:28.574993 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d072e52-69e9-4f25-b4df-c38f94858727 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:28.575456 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d072e52-69e9-4f25-b4df-c38f94858727 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:28.576280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d072e52-69e9-4f25-b4df-c38f94858727 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:28.577905 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d072e52-69e9-4f25-b4df-c38f94858727 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:28.588465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-6d072e52-69e9-4f25-b4df-c38f94858727 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 19:50:28.588465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:28.588465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:28.588465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:28.588465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:28.588465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:28.588465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:28.588465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:28.588465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 19:50:28.588465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:28.588465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:28.588465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:28.588465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:28.588465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:28.588465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:28.588465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:28.590209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:28.590209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:28.590209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:28.590209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:28.590209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:28.590209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:28.590209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:28.590209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:28.590209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:28.590209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:28.590209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:28.590209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:28.590209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:28.590209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:28.590209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:28.590209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:28.591427 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:28.591427 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:28.591427 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:28.591427 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:28.591427 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:28.591427 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:28.591427 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:28.591427 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:28.591427 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:28.591427 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:28.591427 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:28.591427 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:28.591427 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:28.591427 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 19:50:28.591427 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:28.593902 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 388/763] 10.222.0.39 () {56 vars in 1129 bytes} [Fri Aug 5 19:50:28 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:28.600217 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d072e52-69e9-4f25-b4df-c38f94858727 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:28.617712 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 376/764] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:50:28 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:50:28.625692 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5227bad2-2071-4515-afe9-220db895b2cb None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:28.742402 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d072e52-69e9-4f25-b4df-c38f94858727 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:28.751541 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:28.758107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5227bad2-2071-4515-afe9-220db895b2cb None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:28.790126 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9253001-7a5d-455c-ba1c-a8a0fd8d308b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:28.790846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9253001-7a5d-455c-ba1c-a8a0fd8d308b None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:28.791200 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9253001-7a5d-455c-ba1c-a8a0fd8d308b None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:28.791542 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9253001-7a5d-455c-ba1c-a8a0fd8d308b None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:28.792408 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9253001-7a5d-455c-ba1c-a8a0fd8d308b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:28.793369 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9253001-7a5d-455c-ba1c-a8a0fd8d308b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:28.794898 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9253001-7a5d-455c-ba1c-a8a0fd8d308b None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:28.797809 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:28.802266 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 389/765] 10.222.0.39 () {56 vars in 1124 bytes} [Fri Aug 5 19:50:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:28.808313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9253001-7a5d-455c-ba1c-a8a0fd8d308b None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:28.813365 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87404e83-9b6c-4e65-bab6-25b83b9d2976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:28.814277 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87404e83-9b6c-4e65-bab6-25b83b9d2976 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:28.814728 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87404e83-9b6c-4e65-bab6-25b83b9d2976 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:28.815174 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87404e83-9b6c-4e65-bab6-25b83b9d2976 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:28.827652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87404e83-9b6c-4e65-bab6-25b83b9d2976 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:28.830052 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87404e83-9b6c-4e65-bab6-25b83b9d2976 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:28.847579 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-87404e83-9b6c-4e65-bab6-25b83b9d2976 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:28.847579 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:28.847579 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:28.847579 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:28.847579 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:28.847579 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:28.847579 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:28.847579 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:28.847579 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:28.847579 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:28.847579 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:28.847579 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:28.847579 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:28.847579 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:28.847579 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:28.848519 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:28.848519 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:28.848519 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:28.848519 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:28.848519 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:28.848519 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:28.848519 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:28.848519 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:28.848519 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:28.848519 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:28.848519 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:28.848519 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:28.848519 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:28.848519 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:28.848519 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:28.848519 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:28.848519 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:28.849558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:28.849558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:28.849558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:28.849558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:28.851659 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 377/766] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:28.858143 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87404e83-9b6c-4e65-bab6-25b83b9d2976 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:29.023862 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87404e83-9b6c-4e65-bab6-25b83b9d2976 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:29.077198 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a45728-e5cb-410b-be23-96d1b9aa471c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:29.077824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a45728-e5cb-410b-be23-96d1b9aa471c None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:29.078105 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a45728-e5cb-410b-be23-96d1b9aa471c None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:29.078381 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a45728-e5cb-410b-be23-96d1b9aa471c None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:29.079157 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a45728-e5cb-410b-be23-96d1b9aa471c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:29.079466 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a45728-e5cb-410b-be23-96d1b9aa471c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:29.081010 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a45728-e5cb-410b-be23-96d1b9aa471c None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:29.094773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 378/767] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:29.096764 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9253001-7a5d-455c-ba1c-a8a0fd8d308b None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:29.112663 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a45728-e5cb-410b-be23-96d1b9aa471c None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:29.132056 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:29.162493 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9acfdcf-983d-4334-94fc-79e2d2016660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:29.163229 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9acfdcf-983d-4334-94fc-79e2d2016660 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:29.163490 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9acfdcf-983d-4334-94fc-79e2d2016660 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:29.163648 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9acfdcf-983d-4334-94fc-79e2d2016660 None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:29.164343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9acfdcf-983d-4334-94fc-79e2d2016660 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:29.164841 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9acfdcf-983d-4334-94fc-79e2d2016660 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:29.166226 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9acfdcf-983d-4334-94fc-79e2d2016660 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:29.177445 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 390/768] 10.222.0.39 () {56 vars in 1124 bytes} [Fri Aug 5 19:50:28 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:29.184372 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9acfdcf-983d-4334-94fc-79e2d2016660 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:29.322706 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:29.347069 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a45728-e5cb-410b-be23-96d1b9aa471c None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:29.387309 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920b0f7-0b64-4040-b220-c5c6dc5339a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:29.388426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920b0f7-0b64-4040-b220-c5c6dc5339a4 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:29.388861 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920b0f7-0b64-4040-b220-c5c6dc5339a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:29.389130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920b0f7-0b64-4040-b220-c5c6dc5339a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:29.397068 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3920b0f7-0b64-4040-b220-c5c6dc5339a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:29.398788 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3920b0f7-0b64-4040-b220-c5c6dc5339a4 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:29.411634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-3920b0f7-0b64-4040-b220-c5c6dc5339a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:29.411634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:29.411634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:29.411634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:29.411634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:29.411634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:29.411634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:29.411634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:29.411634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:29.411634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:29.411634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:29.411634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:29.411634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:29.411634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:29.411634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:29.413420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:29.413420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:29.413420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:29.413420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:29.413420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:29.413420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:29.413420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:29.413420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:29.413420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:29.413420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:29.413420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:29.413420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:29.413420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:29.413420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:29.413420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:29.413420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:29.413420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:29.415227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:29.415227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:29.415227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:29.415227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:29.415227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:29.415227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:29.415227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:29.415227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:29.417326 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9acfdcf-983d-4334-94fc-79e2d2016660 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:29.418542 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 379/769] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:29.419823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b89287-4d03-42fd-8e0e-add3d301886f None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:29.420114 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b89287-4d03-42fd-8e0e-add3d301886f None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:29.420376 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b89287-4d03-42fd-8e0e-add3d301886f None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:29.428019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 380/770] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:29.428019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b89287-4d03-42fd-8e0e-add3d301886f None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:29.478355 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d5d639-c928-4baa-b277-201c8d9f13c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:29.479730 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d5d639-c928-4baa-b277-201c8d9f13c6 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:29.480777 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d5d639-c928-4baa-b277-201c8d9f13c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:29.481779 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d5d639-c928-4baa-b277-201c8d9f13c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:29.482731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d5d639-c928-4baa-b277-201c8d9f13c6 None admin] RBAC: Adding query filter params (interface=public, service_id=1b8ffa1d024c4a08be70276eac6d72ce, region_id=RegionOne) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:29.483446 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d5d639-c928-4baa-b277-201c8d9f13c6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:29.485474 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d5d639-c928-4baa-b277-201c8d9f13c6 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:29.500851 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 391/771] 10.222.0.39 () {56 vars in 1251 bytes} [Fri Aug 5 19:50:29 2022] GET /identity/v3/endpoints?service_id=1b8ffa1d024c4a08be70276eac6d72ce&interface=public®ion_id=RegionOne => generated 194 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:29.500851 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e4bca-8b1b-41cd-a9d4-c77353986aee None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:29.500851 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e4bca-8b1b-41cd-a9d4-c77353986aee None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:29.500851 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e4bca-8b1b-41cd-a9d4-c77353986aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:29.512361 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:29.528565 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-f02e4bca-8b1b-41cd-a9d4-c77353986aee None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:29.531055 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-f02e4bca-8b1b-41cd-a9d4-c77353986aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:29.567466 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b89287-4d03-42fd-8e0e-add3d301886f None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:29.614662 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981cc177-1805-473e-a2ae-67cd62ceabac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:29.615968 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981cc177-1805-473e-a2ae-67cd62ceabac None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:29.616611 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981cc177-1805-473e-a2ae-67cd62ceabac None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:29.617473 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981cc177-1805-473e-a2ae-67cd62ceabac None admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:29.619889 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981cc177-1805-473e-a2ae-67cd62ceabac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:29.620858 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981cc177-1805-473e-a2ae-67cd62ceabac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:29.624361 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981cc177-1805-473e-a2ae-67cd62ceabac None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:29.632682 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 392/772] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:29.632682 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e4bca-8b1b-41cd-a9d4-c77353986aee None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:29.651333 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 381/773] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:29.658856 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981cc177-1805-473e-a2ae-67cd62ceabac None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:29.730773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:29.786065 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e4bca-8b1b-41cd-a9d4-c77353986aee None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:29.795878 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981cc177-1805-473e-a2ae-67cd62ceabac None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:29.827232 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a28bb5f-aea8-45a3-9c03-fb3b4f94838c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:29.827850 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28bb5f-aea8-45a3-9c03-fb3b4f94838c None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:29.828126 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28bb5f-aea8-45a3-9c03-fb3b4f94838c None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:29.828397 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28bb5f-aea8-45a3-9c03-fb3b4f94838c None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:29.829098 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a28bb5f-aea8-45a3-9c03-fb3b4f94838c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:29.830639 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a28bb5f-aea8-45a3-9c03-fb3b4f94838c None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:29.836679 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 393/774] 10.222.0.39 () {56 vars in 1087 bytes} [Fri Aug 5 19:50:29 2022] GET /identity/v3/services => generated 772 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:29.837530 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:29.841597 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a28bb5f-aea8-45a3-9c03-fb3b4f94838c None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:29.849156 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2281f9d-ca4d-4be2-a606-94d7c538d861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:29.851242 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2281f9d-ca4d-4be2-a606-94d7c538d861 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:29.851785 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2281f9d-ca4d-4be2-a606-94d7c538d861 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:29.852236 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2281f9d-ca4d-4be2-a606-94d7c538d861 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:29.853258 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2281f9d-ca4d-4be2-a606-94d7c538d861 None admin] RBAC: Adding query filter params (role.id=7ebfd9e58fe549f08cdaec5f906a38a6, scope.project.id=80379e2104644e98a2e85d4e4af1b365, user.id=3488a7d2258242c0859a105c1d162767) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:29.853739 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2281f9d-ca4d-4be2-a606-94d7c538d861 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:29.856272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2281f9d-ca4d-4be2-a606-94d7c538d861 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:29.865230 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 382/775] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:50:29 2022] GET /identity/v3/role_assignments?user.id=3488a7d2258242c0859a105c1d162767&scope.project.id=80379e2104644e98a2e85d4e4af1b365&role.id=7ebfd9e58fe549f08cdaec5f906a38a6 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:29.986112 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a28bb5f-aea8-45a3-9c03-fb3b4f94838c None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:30.032929 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:30.053479 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca8e14-6c3f-436e-b743-bf575c51c00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:30.054161 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca8e14-6c3f-436e-b743-bf575c51c00a None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:30.054448 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca8e14-6c3f-436e-b743-bf575c51c00a None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:30.054719 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca8e14-6c3f-436e-b743-bf575c51c00a None admin] PATH_INFO: `/v3/services/network` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:30.055368 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ca8e14-6c3f-436e-b743-bf575c51c00a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:30.056905 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ca8e14-6c3f-436e-b743-bf575c51c00a None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:30.068070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-a5ca8e14-6c3f-436e-b743-bf575c51c00a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 19:50:30.068070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:30.068070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:30.068070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:30.068070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:30.068070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:30.068070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:30.068070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:30.068070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 19:50:30.068070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:30.068070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:30.068070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:30.068070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:30.068070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:30.068070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:30.068070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:30.068070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:30.069404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:30.069404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:30.069404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:30.069404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:30.069404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:30.069404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:30.069404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:30.069404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:30.069404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:30.069404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:30.069404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:30.069404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:30.069404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:30.069404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:30.069404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:30.069404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:30.070634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:30.070634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:30.070634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:30.070634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:30.070634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:30.070634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:30.070634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:30.070634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:30.070634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:30.070634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:30.070634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:30.070634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:30.070634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 19:50:30.070634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:30.070634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 394/776] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:29 2022] GET /identity/v3/services/network => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:30.076092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2281f9d-ca4d-4be2-a606-94d7c538d861 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:30.200425 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2281f9d-ca4d-4be2-a606-94d7c538d861 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:30.240517 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35974e3-9eb7-44ed-b976-802c9de9b3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:30.241665 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35974e3-9eb7-44ed-b976-802c9de9b3ca None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:30.242100 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35974e3-9eb7-44ed-b976-802c9de9b3ca None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:30.242546 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35974e3-9eb7-44ed-b976-802c9de9b3ca None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:30.243388 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35974e3-9eb7-44ed-b976-802c9de9b3ca None admin] RBAC: Adding query filter params (name=network) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:30.243841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35974e3-9eb7-44ed-b976-802c9de9b3ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:30.245394 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35974e3-9eb7-44ed-b976-802c9de9b3ca None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:30.251210 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 383/777] 10.222.0.39 () {56 vars in 1112 bytes} [Fri Aug 5 19:50:30 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:30.257408 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca8e14-6c3f-436e-b743-bf575c51c00a None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:30.420490 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca8e14-6c3f-436e-b743-bf575c51c00a None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:30.474193 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186db56-64e8-4615-b4ff-dc7e5bec4b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:30.474815 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186db56-64e8-4615-b4ff-dc7e5bec4b57 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:30.475095 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186db56-64e8-4615-b4ff-dc7e5bec4b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:30.475366 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186db56-64e8-4615-b4ff-dc7e5bec4b57 None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:30.476046 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186db56-64e8-4615-b4ff-dc7e5bec4b57 None admin] RBAC: Adding query filter params (type=network) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:30.476351 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186db56-64e8-4615-b4ff-dc7e5bec4b57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:30.477753 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186db56-64e8-4615-b4ff-dc7e5bec4b57 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:30.489423 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 395/778] 10.222.0.39 () {56 vars in 1112 bytes} [Fri Aug 5 19:50:30 2022] GET /identity/v3/services?type=network => generated 350 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:30.494781 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35974e3-9eb7-44ed-b976-802c9de9b3ca None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:30.603368 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:30.659350 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35974e3-9eb7-44ed-b976-802c9de9b3ca None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:30.709858 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c84b5-1776-46cd-a1e7-87972e831536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:30.710470 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c84b5-1776-46cd-a1e7-87972e831536 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:30.710745 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c84b5-1776-46cd-a1e7-87972e831536 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:30.711015 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c84b5-1776-46cd-a1e7-87972e831536 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:30.711793 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c84b5-1776-46cd-a1e7-87972e831536 None admin] RBAC: Adding query filter params (interface=public, service_id=78ccc5cbe80647009693d1fe5990d51a, region_id=RegionOne) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:30.712101 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c84b5-1776-46cd-a1e7-87972e831536 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:30.713580 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c84b5-1776-46cd-a1e7-87972e831536 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:30.726521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 384/779] 10.222.0.39 () {56 vars in 1251 bytes} [Fri Aug 5 19:50:30 2022] GET /identity/v3/endpoints?service_id=78ccc5cbe80647009693d1fe5990d51a&interface=public®ion_id=RegionOne => generated 194 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:30.732756 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186db56-64e8-4615-b4ff-dc7e5bec4b57 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:30.821414 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af0505-0dd9-414b-be5c-d45f02dc58d9 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:30.821829 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af0505-0dd9-414b-be5c-d45f02dc58d9 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:30.822003 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af0505-0dd9-414b-be5c-d45f02dc58d9 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:30.823619 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 385/780] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:30.828701 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:30.838275 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b90e9e-03e4-497b-bb7e-941691a35cd2 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:30.838275 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b90e9e-03e4-497b-bb7e-941691a35cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:30.838669 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b90e9e-03e4-497b-bb7e-941691a35cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:30.882218 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-72b90e9e-03e4-497b-bb7e-941691a35cd2 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:30.883423 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-72b90e9e-03e4-497b-bb7e-941691a35cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:30.890652 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186db56-64e8-4615-b4ff-dc7e5bec4b57 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:30.929792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa76e1-4af0-45f3-80ff-434a2f4819ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:30.930771 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa76e1-4af0-45f3-80ff-434a2f4819ad None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:30.931256 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa76e1-4af0-45f3-80ff-434a2f4819ad None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:30.931752 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa76e1-4af0-45f3-80ff-434a2f4819ad None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:30.932747 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fa76e1-4af0-45f3-80ff-434a2f4819ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:30.934547 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fa76e1-4af0-45f3-80ff-434a2f4819ad None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:30.942054 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 396/781] 10.222.0.39 () {56 vars in 1087 bytes} [Fri Aug 5 19:50:30 2022] GET /identity/v3/services => generated 772 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:30.991120 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 386/782] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:31.012618 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52f959-f5e0-4f30-a03f-28af8dddba69 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:31.013047 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52f959-f5e0-4f30-a03f-28af8dddba69 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:31.013329 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52f959-f5e0-4f30-a03f-28af8dddba69 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:31.015128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 397/783] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:31.020270 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b90e9e-03e4-497b-bb7e-941691a35cd2 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:31.020882 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e94d2-8951-4bb9-9478-0ff9e822a44e None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:31.021172 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e94d2-8951-4bb9-9478-0ff9e822a44e None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:31.021452 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e94d2-8951-4bb9-9478-0ff9e822a44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:31.059311 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-ad2e94d2-8951-4bb9-9478-0ff9e822a44e None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:31.061219 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-ad2e94d2-8951-4bb9-9478-0ff9e822a44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:31.080000 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:31.185828 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 398/784] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:31.199454 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b90e9e-03e4-497b-bb7e-941691a35cd2 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:31.216142 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e94d2-8951-4bb9-9478-0ff9e822a44e None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:31.254283 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd95cbc-c89a-46f9-9993-0307e4dd2ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:31.255501 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd95cbc-c89a-46f9-9993-0307e4dd2ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:31.255937 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd95cbc-c89a-46f9-9993-0307e4dd2ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:31.256353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd95cbc-c89a-46f9-9993-0307e4dd2ec1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:31.269487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:31.272097 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd95cbc-c89a-46f9-9993-0307e4dd2ec1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:31.279117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-6bd95cbc-c89a-46f9-9993-0307e4dd2ec1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:50:31.279117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:31.279117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:31.279117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:31.279117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:31.279117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:31.279117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:31.279117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:31.279117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:31.279117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:31.279117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:31.279117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:31.279117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:31.279117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:31.279117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:31.279117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:31.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:31.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:31.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:31.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:31.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:31.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:31.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:31.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:31.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:31.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:31.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:31.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:31.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:31.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:31.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:31.280257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:31.281303 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:31.281303 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:31.281303 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:31.281303 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:31.281303 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:31.281303 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:31.281303 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:31.281303 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:31.281303 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:31.281303 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:31.281303 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:50:31.281303 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:31.283543 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 387/785] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:50:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:31.288884 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd95cbc-c89a-46f9-9993-0307e4dd2ec1 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:31.338343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e94d2-8951-4bb9-9478-0ff9e822a44e None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:31.400821 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd95cbc-c89a-46f9-9993-0307e4dd2ec1 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:31.407706 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d903f-4cba-4167-a72b-da1b293ca9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:31.408364 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d903f-4cba-4167-a72b-da1b293ca9dc None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:31.408671 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d903f-4cba-4167-a72b-da1b293ca9dc None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:31.408950 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d903f-4cba-4167-a72b-da1b293ca9dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:31.415326 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:31.437746 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867d903f-4cba-4167-a72b-da1b293ca9dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:31.443987 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e48ba-8206-4f6e-97b6-c4e75379897c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:31.444834 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e48ba-8206-4f6e-97b6-c4e75379897c None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:31.445264 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e48ba-8206-4f6e-97b6-c4e75379897c None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:31.445676 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e48ba-8206-4f6e-97b6-c4e75379897c None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:31.445813 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-867d903f-4cba-4167-a72b-da1b293ca9dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:50:31.445813 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:31.445813 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:31.445813 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:31.445813 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:31.445813 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:31.445813 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:31.445813 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:31.445813 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:31.445813 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:31.445813 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:31.445813 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:31.445813 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:31.445813 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:31.445813 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:31.445813 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:31.446889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:31.446889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:31.446889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:31.446889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:31.446889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:31.446889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:31.446889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:31.446889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:31.446889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:31.446889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:31.446889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:31.446889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:31.446889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:31.446889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:31.446889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:31.446889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:31.447859 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:31.447859 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:31.447859 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:31.447859 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:31.447859 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:31.447859 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:31.447859 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:31.447859 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:31.447859 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:31.447859 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:31.447859 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:50:31.447859 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:31.450148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91e48ba-8206-4f6e-97b6-c4e75379897c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:31.450627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91e48ba-8206-4f6e-97b6-c4e75379897c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:31.452145 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91e48ba-8206-4f6e-97b6-c4e75379897c None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:31.453102 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 399/786] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:50:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:31.459336 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d903f-4cba-4167-a72b-da1b293ca9dc None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:31.462758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 388/787] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:50:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:31.471981 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e48ba-8206-4f6e-97b6-c4e75379897c None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:31.622076 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d903f-4cba-4167-a72b-da1b293ca9dc None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:31.624523 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:31.666518 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e48ba-8206-4f6e-97b6-c4e75379897c None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:31.670796 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfc582-05ef-40d7-abbc-c28366619196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:31.671425 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfc582-05ef-40d7-abbc-c28366619196 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:31.671704 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfc582-05ef-40d7-abbc-c28366619196 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:31.671978 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfc582-05ef-40d7-abbc-c28366619196 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:31.672740 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcfc582-05ef-40d7-abbc-c28366619196 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:31.673050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcfc582-05ef-40d7-abbc-c28366619196 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:31.674436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcfc582-05ef-40d7-abbc-c28366619196 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:31.693010 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 400/788] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:50:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:31.699819 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfc582-05ef-40d7-abbc-c28366619196 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:31.740821 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab8b4c-47f9-4915-a17f-9b586425cb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:31.741467 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab8b4c-47f9-4915-a17f-9b586425cb60 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:31.741748 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab8b4c-47f9-4915-a17f-9b586425cb60 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:31.742021 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab8b4c-47f9-4915-a17f-9b586425cb60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:31.759072 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffab8b4c-47f9-4915-a17f-9b586425cb60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:31.760523 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffab8b4c-47f9-4915-a17f-9b586425cb60 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:31.792694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-ffab8b4c-47f9-4915-a17f-9b586425cb60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:31.792694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:31.792694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:31.792694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:31.792694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:31.792694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:31.792694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:31.792694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:31.792694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:31.792694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:31.792694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:31.792694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:31.792694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:31.792694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:31.792694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:31.793846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:31.793846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:31.793846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:31.793846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:31.793846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:31.793846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:31.793846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:31.793846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:31.793846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:31.793846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:31.793846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:31.793846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:31.793846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:31.793846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:31.793846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:31.793846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:31.793846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:31.794753 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:31.794753 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:31.794753 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:31.794753 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:31.798883 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 389/789] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:31.800148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab8b4c-47f9-4915-a17f-9b586425cb60 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:31.830379 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:31.908500 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfc582-05ef-40d7-abbc-c28366619196 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:31.915175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab8b4c-47f9-4915-a17f-9b586425cb60 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:31.952301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b506f-7850-4867-bdf7-8fe3f0f88846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:31.952953 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b506f-7850-4867-bdf7-8fe3f0f88846 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:31.953234 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b506f-7850-4867-bdf7-8fe3f0f88846 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:31.953511 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b506f-7850-4867-bdf7-8fe3f0f88846 None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:31.954206 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601b506f-7850-4867-bdf7-8fe3f0f88846 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:31.954535 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601b506f-7850-4867-bdf7-8fe3f0f88846 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:31.960607 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601b506f-7850-4867-bdf7-8fe3f0f88846 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:31.973691 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 390/790] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:31.975016 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b506f-7850-4867-bdf7-8fe3f0f88846 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:31.975820 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee886195-f3c7-427a-9e4a-96337d3bce18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:31.976740 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee886195-f3c7-427a-9e4a-96337d3bce18 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:31.977255 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee886195-f3c7-427a-9e4a-96337d3bce18 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:31.977736 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee886195-f3c7-427a-9e4a-96337d3bce18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:32.004445 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee886195-f3c7-427a-9e4a-96337d3bce18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:32.006270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee886195-f3c7-427a-9e4a-96337d3bce18 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:32.035238 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-ee886195-f3c7-427a-9e4a-96337d3bce18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:32.035238 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:32.035238 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:32.035238 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:32.035238 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:32.035238 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:32.035238 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:32.035238 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:32.035238 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:32.035238 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:32.035238 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:32.035238 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:32.035238 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:32.035238 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:32.035238 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:32.036194 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:32.036194 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:32.036194 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:32.036194 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:32.036194 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:32.036194 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:32.036194 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:32.036194 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:32.036194 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:32.036194 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:32.036194 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:32.036194 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:32.036194 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:32.036194 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:32.036194 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:32.036194 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:32.036194 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:32.037113 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:32.037113 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:32.037113 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:32.037113 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:32.052906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 401/791] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:32.068631 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee886195-f3c7-427a-9e4a-96337d3bce18 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:32.069979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:32.088947 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b506f-7850-4867-bdf7-8fe3f0f88846 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:32.105041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:32.127389 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1049c04-c1de-4b15-a67b-4e7b66cad5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:32.128031 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1049c04-c1de-4b15-a67b-4e7b66cad5d9 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:32.128315 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1049c04-c1de-4b15-a67b-4e7b66cad5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:32.128610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1049c04-c1de-4b15-a67b-4e7b66cad5d9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:32.147962 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1049c04-c1de-4b15-a67b-4e7b66cad5d9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:32.149719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1049c04-c1de-4b15-a67b-4e7b66cad5d9 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:32.176840 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-e1049c04-c1de-4b15-a67b-4e7b66cad5d9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:50:32.176840 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:32.176840 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:32.176840 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:32.176840 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:32.176840 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:32.176840 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:32.176840 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:32.176840 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:32.176840 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:32.176840 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:32.176840 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:32.176840 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:32.176840 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:32.176840 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:32.176840 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:32.177824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:32.177824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:32.177824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:32.177824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:32.177824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:32.177824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:32.177824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:32.177824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:32.177824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:32.177824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:32.177824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:32.177824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:32.177824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:32.177824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:32.177824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:32.177824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:32.177824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:32.177824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:32.178765 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:32.178765 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:32.178765 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:32.178765 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:32.178765 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:32.178765 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:32.178765 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:32.178765 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:32.178765 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:50:32.178765 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:32.184432 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 391/792] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:50:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:32.187784 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1049c04-c1de-4b15-a67b-4e7b66cad5d9 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:32.246643 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee886195-f3c7-427a-9e4a-96337d3bce18 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:32.282808 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dcbf8d-25ef-41de-b2a9-b5600d979c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:32.283473 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcbf8d-25ef-41de-b2a9-b5600d979c46 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:32.283738 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcbf8d-25ef-41de-b2a9-b5600d979c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:32.283970 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcbf8d-25ef-41de-b2a9-b5600d979c46 None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:32.284700 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dcbf8d-25ef-41de-b2a9-b5600d979c46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:32.285222 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dcbf8d-25ef-41de-b2a9-b5600d979c46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:32.286356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dcbf8d-25ef-41de-b2a9-b5600d979c46 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:32.299359 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 402/793] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:32.300804 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b53bfc-a6e7-4bdc-9c53-7b80d29c29b3 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:32.301096 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b53bfc-a6e7-4bdc-9c53-7b80d29c29b3 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:32.301365 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b53bfc-a6e7-4bdc-9c53-7b80d29c29b3 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:32.307399 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 403/794] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:32.307399 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b53bfc-a6e7-4bdc-9c53-7b80d29c29b3 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:32.337684 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1049c04-c1de-4b15-a67b-4e7b66cad5d9 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:32.378326 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b03f59-faa9-4fce-86b6-3bae2bd1857a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:32.379430 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b03f59-faa9-4fce-86b6-3bae2bd1857a None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:32.379913 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b03f59-faa9-4fce-86b6-3bae2bd1857a None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:32.380436 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b03f59-faa9-4fce-86b6-3bae2bd1857a None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:32.381525 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b03f59-faa9-4fce-86b6-3bae2bd1857a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:32.382036 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b03f59-faa9-4fce-86b6-3bae2bd1857a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:32.383778 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b03f59-faa9-4fce-86b6-3bae2bd1857a None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:32.387975 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:32.406883 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 392/795] 10.222.0.39 () {56 vars in 1135 bytes} [Fri Aug 5 19:50:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:32.408589 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da038ee-3912-490a-9941-57fa2af5eefe None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:32.409141 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da038ee-3912-490a-9941-57fa2af5eefe None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:32.409768 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da038ee-3912-490a-9941-57fa2af5eefe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:32.416331 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:32.419987 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b53bfc-a6e7-4bdc-9c53-7b80d29c29b3 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:32.435066 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-8da038ee-3912-490a-9941-57fa2af5eefe None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:32.437205 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-8da038ee-3912-490a-9941-57fa2af5eefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:32.463248 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8ffd1-07a6-496d-8650-2bf251edcb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:32.464644 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8ffd1-07a6-496d-8650-2bf251edcb34 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:32.465026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8ffd1-07a6-496d-8650-2bf251edcb34 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:32.465329 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8ffd1-07a6-496d-8650-2bf251edcb34 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:32.475545 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe8ffd1-07a6-496d-8650-2bf251edcb34 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:32.477262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe8ffd1-07a6-496d-8650-2bf251edcb34 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:32.492277 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-bbe8ffd1-07a6-496d-8650-2bf251edcb34 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:50:32.492277 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:32.492277 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:32.492277 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:32.492277 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:32.492277 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:32.492277 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:32.492277 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:32.492277 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:32.492277 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:32.492277 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:32.492277 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:32.492277 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:32.492277 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:32.492277 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:32.492277 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:32.493324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:32.493324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:32.493324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:32.493324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:32.493324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:32.493324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:32.493324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:32.493324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:32.493324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:32.493324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:32.493324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:32.493324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:32.493324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:32.493324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:32.493324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:32.493324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:32.493324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:32.493324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:32.495973 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:32.495973 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:32.495973 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:32.495973 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:32.495973 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:32.495973 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:32.495973 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:32.495973 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:32.495973 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:50:32.495973 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:32.495973 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 404/796] 10.222.0.39 () {56 vars in 1093 bytes} [Fri Aug 5 19:50:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:32.495973 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8ffd1-07a6-496d-8650-2bf251edcb34 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:32.535727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 393/797] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:32.537369 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da038ee-3912-490a-9941-57fa2af5eefe None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:32.611973 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8ffd1-07a6-496d-8650-2bf251edcb34 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:32.649446 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e52ad-c9f4-4631-87ba-bce8ce49fe9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:32.650082 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e52ad-c9f4-4631-87ba-bce8ce49fe9e None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:32.650362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e52ad-c9f4-4631-87ba-bce8ce49fe9e None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:32.650639 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e52ad-c9f4-4631-87ba-bce8ce49fe9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:32.658536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:32.659090 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da038ee-3912-490a-9941-57fa2af5eefe None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:32.663059 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e52ad-c9f4-4631-87ba-bce8ce49fe9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:32.664609 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e52ad-c9f4-4631-87ba-bce8ce49fe9e None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:32.680746 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-959e52ad-c9f4-4631-87ba-bce8ce49fe9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:32.680746 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:32.680746 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:32.680746 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:32.680746 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:32.680746 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:32.680746 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:32.680746 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:32.680746 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:32.680746 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:32.680746 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:32.680746 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:32.680746 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:32.680746 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:32.680746 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:32.681681 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:32.681681 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:32.681681 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:32.681681 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:32.681681 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:32.681681 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:32.681681 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:32.681681 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:32.681681 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:32.681681 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:32.681681 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:32.681681 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:32.681681 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:32.681681 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:32.681681 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:32.681681 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:32.681681 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:32.682567 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:32.682567 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:32.682567 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:32.682567 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:32.686982 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 405/798] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:32.688204 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83750a17-319d-492a-9723-74ea9aa02091 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:32.688499 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83750a17-319d-492a-9723-74ea9aa02091 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:32.688807 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83750a17-319d-492a-9723-74ea9aa02091 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:32.690079 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 406/799] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:32.690760 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83750a17-319d-492a-9723-74ea9aa02091 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:32.695391 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:32.701577 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314af97d-00fb-4538-a4e5-13261e77c707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:32.702205 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314af97d-00fb-4538-a4e5-13261e77c707 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:32.702489 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314af97d-00fb-4538-a4e5-13261e77c707 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:32.702779 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314af97d-00fb-4538-a4e5-13261e77c707 None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:32.703560 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314af97d-00fb-4538-a4e5-13261e77c707 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:32.703871 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314af97d-00fb-4538-a4e5-13261e77c707 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:32.705334 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314af97d-00fb-4538-a4e5-13261e77c707 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:32.723965 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 394/800] 10.222.0.39 () {56 vars in 1133 bytes} [Fri Aug 5 19:50:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:32.725002 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314af97d-00fb-4538-a4e5-13261e77c707 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:32.806838 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83750a17-319d-492a-9723-74ea9aa02091 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:32.830788 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314af97d-00fb-4538-a4e5-13261e77c707 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:32.845682 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4f37d-4455-4f36-93ac-54470e967e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:32.846514 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4f37d-4455-4f36-93ac-54470e967e54 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:32.846651 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4f37d-4455-4f36-93ac-54470e967e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:32.846873 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4f37d-4455-4f36-93ac-54470e967e54 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:32.859147 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac4f37d-4455-4f36-93ac-54470e967e54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:32.864097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-9ac4f37d-4455-4f36-93ac-54470e967e54 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:32.864097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:32.864097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:32.864097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:32.864097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:32.864097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:32.864097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:32.864097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:32.864097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:32.864097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:32.864097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:32.864097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:32.864097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:32.864097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:32.864097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:32.864097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:32.865410 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:32.865410 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:32.865410 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:32.865410 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:32.865410 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:32.865410 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:32.865410 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:32.865410 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:32.865410 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:32.865410 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:32.865410 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:32.865410 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:32.865410 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:32.865410 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:32.865410 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:32.865410 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:32.866478 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:32.866478 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:32.866478 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:32.866478 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:32.866478 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:32.866478 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:32.866478 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:32.866478 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:32.866478 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:32.866478 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:32.866478 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:32.866478 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:32.871014 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 407/801] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Aug 5 19:50:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:32.872384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce41984-f5dd-429c-bf3d-81e35195a0ca None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:32.872702 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce41984-f5dd-429c-bf3d-81e35195a0ca None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:32.872982 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce41984-f5dd-429c-bf3d-81e35195a0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:32.873622 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da0cb7-2d67-4c5c-9c2b-f1672b64336b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:32.874416 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da0cb7-2d67-4c5c-9c2b-f1672b64336b None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:32.874851 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da0cb7-2d67-4c5c-9c2b-f1672b64336b None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:32.875324 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da0cb7-2d67-4c5c-9c2b-f1672b64336b None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:32.876391 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da0cb7-2d67-4c5c-9c2b-f1672b64336b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:32.876960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da0cb7-2d67-4c5c-9c2b-f1672b64336b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:32.878569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da0cb7-2d67-4c5c-9c2b-f1672b64336b None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:32.891221 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 395/802] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:32.892393 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da0cb7-2d67-4c5c-9c2b-f1672b64336b None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:32.893307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-8ce41984-f5dd-429c-bf3d-81e35195a0ca None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:32.894490 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-8ce41984-f5dd-429c-bf3d-81e35195a0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:32.947237 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:32.953504 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:32.972191 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 408/803] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:32.973392 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce41984-f5dd-429c-bf3d-81e35195a0ca None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:33.021503 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da0cb7-2d67-4c5c-9c2b-f1672b64336b None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:33.061012 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b05c3-4b42-420c-b36d-b0d09b789c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:33.061663 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b05c3-4b42-420c-b36d-b0d09b789c04 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:33.061906 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b05c3-4b42-420c-b36d-b0d09b789c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:33.062179 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b05c3-4b42-420c-b36d-b0d09b789c04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:33.085828 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce41984-f5dd-429c-bf3d-81e35195a0ca None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:33.131099 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a4ac3-729a-4e6f-80ed-65477b2e21d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:33.131631 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a4ac3-729a-4e6f-80ed-65477b2e21d4 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:33.131911 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a4ac3-729a-4e6f-80ed-65477b2e21d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:33.132181 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a4ac3-729a-4e6f-80ed-65477b2e21d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:33.132940 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a4ac3-729a-4e6f-80ed-65477b2e21d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:33.133253 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a4ac3-729a-4e6f-80ed-65477b2e21d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:33.134623 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a4ac3-729a-4e6f-80ed-65477b2e21d4 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:33.142562 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b05c3-4b42-420c-b36d-b0d09b789c04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:33.142562 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b05c3-4b42-420c-b36d-b0d09b789c04 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:33.152420 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 409/804] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Aug 5 19:50:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:33.152420 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a4ac3-729a-4e6f-80ed-65477b2e21d4 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:33.196211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:33.196411 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:33.256723 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 396/805] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:50:32 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:50:33.257864 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b05c3-4b42-420c-b36d-b0d09b789c04 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:33.265338 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a4ac3-729a-4e6f-80ed-65477b2e21d4 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:33.300130 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453adb6-b930-4dc9-9e0c-043e813ed045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:33.300796 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453adb6-b930-4dc9-9e0c-043e813ed045 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:33.301073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453adb6-b930-4dc9-9e0c-043e813ed045 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:33.301349 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453adb6-b930-4dc9-9e0c-043e813ed045 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:33.309137 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4453adb6-b930-4dc9-9e0c-043e813ed045 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:33.311605 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4453adb6-b930-4dc9-9e0c-043e813ed045 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:33.325330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-4453adb6-b930-4dc9-9e0c-043e813ed045 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:33.325330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:33.325330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:33.325330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:33.325330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:33.325330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:33.325330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:33.325330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:33.325330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:33.325330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:33.325330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:33.325330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:33.325330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:33.325330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:33.325330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:33.326426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:33.326426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:33.326426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:33.326426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:33.326426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:33.326426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:33.326426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:33.326426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:33.326426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:33.326426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:33.326426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:33.326426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:33.326426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:33.326426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:33.326426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:33.326426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:33.326426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:33.327736 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:33.327736 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:33.327736 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:33.327736 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:33.327736 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:33.327736 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:33.327736 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:33.327736 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:33.331310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 410/806] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:33.331845 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453adb6-b930-4dc9-9e0c-043e813ed045 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:33.381583 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b05c3-4b42-420c-b36d-b0d09b789c04 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:33.417286 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a8ba99-0bbe-4cc0-b5be-e76d77815b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:33.417971 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8ba99-0bbe-4cc0-b5be-e76d77815b07 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:33.418251 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8ba99-0bbe-4cc0-b5be-e76d77815b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:33.418523 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8ba99-0bbe-4cc0-b5be-e76d77815b07 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:33.419142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a8ba99-0bbe-4cc0-b5be-e76d77815b07 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:33.420494 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a8ba99-0bbe-4cc0-b5be-e76d77815b07 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:33.429357 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-e6a8ba99-0bbe-4cc0-b5be-e76d77815b07 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 19:50:33.429357 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:33.429357 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:33.429357 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:33.429357 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:33.429357 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:33.429357 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:33.429357 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:33.429357 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 19:50:33.429357 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:33.429357 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:33.429357 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:33.429357 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:33.429357 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:33.429357 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:33.429357 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:33.430366 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:33.430366 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:33.430366 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:33.430366 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:33.430366 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:33.430366 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:33.430366 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:33.430366 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:33.430366 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:33.430366 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:33.430366 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:33.430366 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:33.430366 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:33.430366 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:33.430366 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:33.430366 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:33.431216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:33.431216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:33.431216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:33.431216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:33.431216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:33.431216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:33.431216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:33.431216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:33.431216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:33.431216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:33.431216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:33.431216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:33.431216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:33.431216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 19:50:33.431216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:33.431216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 397/807] 10.222.0.39 () {56 vars in 1129 bytes} [Fri Aug 5 19:50:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:33.432075 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a8ba99-0bbe-4cc0-b5be-e76d77815b07 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:33.446440 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453adb6-b930-4dc9-9e0c-043e813ed045 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:33.447048 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:33.447981 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:33.482815 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e6e2f-1eb7-41cc-a024-1df9bf6e13f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:33.483447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e6e2f-1eb7-41cc-a024-1df9bf6e13f4 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:33.483734 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e6e2f-1eb7-41cc-a024-1df9bf6e13f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:33.484007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e6e2f-1eb7-41cc-a024-1df9bf6e13f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:33.494699 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e6e2f-1eb7-41cc-a024-1df9bf6e13f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:33.496722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e6e2f-1eb7-41cc-a024-1df9bf6e13f4 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:33.510788 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-3c5e6e2f-1eb7-41cc-a024-1df9bf6e13f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:33.510788 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:33.510788 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:33.510788 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:33.510788 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:33.510788 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:33.510788 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:33.510788 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:33.510788 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:33.510788 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:33.510788 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:33.510788 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:33.510788 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:33.510788 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:33.510788 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:33.512300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:33.512300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:33.512300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:33.512300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:33.512300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:33.512300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:33.512300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:33.512300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:33.512300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:33.512300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:33.512300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:33.512300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:33.512300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:33.512300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:33.512300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:33.512300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:33.512300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:33.518253 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:33.518253 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:33.518253 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:33.518253 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:33.518253 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 411/808] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:33.518253 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e6e2f-1eb7-41cc-a024-1df9bf6e13f4 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:33.533393 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a8ba99-0bbe-4cc0-b5be-e76d77815b07 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:33.566569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189ee05-1ef8-4795-8f72-81fab2337818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:33.567202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189ee05-1ef8-4795-8f72-81fab2337818 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:33.567479 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189ee05-1ef8-4795-8f72-81fab2337818 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:33.567748 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189ee05-1ef8-4795-8f72-81fab2337818 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:33.578946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5189ee05-1ef8-4795-8f72-81fab2337818 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:33.580388 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5189ee05-1ef8-4795-8f72-81fab2337818 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:33.595121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-5189ee05-1ef8-4795-8f72-81fab2337818 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:33.595121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:33.595121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:33.595121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:33.595121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:33.595121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:33.595121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:33.595121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:33.595121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:33.595121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:33.595121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:33.595121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:33.595121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:33.595121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:33.595121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:33.596058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:33.596058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:33.596058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:33.596058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:33.596058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:33.596058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:33.596058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:33.596058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:33.596058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:33.596058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:33.596058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:33.596058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:33.596058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:33.596058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:33.596058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:33.596058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:33.596058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:33.596967 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:33.596967 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:33.596967 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:33.596967 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:33.596967 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 398/809] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:33.597667 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acb87a-1e82-4dbf-8592-a8c5fa06345b None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:33.597925 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acb87a-1e82-4dbf-8592-a8c5fa06345b None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:33.598196 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acb87a-1e82-4dbf-8592-a8c5fa06345b None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:33.599414 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 399/810] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:33.600101 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7acb87a-1e82-4dbf-8592-a8c5fa06345b None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:33.623407 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e6e2f-1eb7-41cc-a024-1df9bf6e13f4 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:33.658548 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a315460-3dd2-47b7-ab79-11b555dbc0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:33.659164 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a315460-3dd2-47b7-ab79-11b555dbc0d4 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:33.659443 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a315460-3dd2-47b7-ab79-11b555dbc0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:33.659713 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a315460-3dd2-47b7-ab79-11b555dbc0d4 None admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:33.660459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a315460-3dd2-47b7-ab79-11b555dbc0d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:33.660792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a315460-3dd2-47b7-ab79-11b555dbc0d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:33.662373 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a315460-3dd2-47b7-ab79-11b555dbc0d4 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:33.670304 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:33.674884 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 412/811] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:33.675688 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a315460-3dd2-47b7-ab79-11b555dbc0d4 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:33.691484 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:33.712679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7acb87a-1e82-4dbf-8592-a8c5fa06345b None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:33.747172 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d947b0-c0a5-4cfd-9d06-b64e28687cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:33.747784 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d947b0-c0a5-4cfd-9d06-b64e28687cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:33.748065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d947b0-c0a5-4cfd-9d06-b64e28687cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:33.748342 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d947b0-c0a5-4cfd-9d06-b64e28687cb0 None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:33.749127 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d947b0-c0a5-4cfd-9d06-b64e28687cb0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:33.749434 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d947b0-c0a5-4cfd-9d06-b64e28687cb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:33.750798 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d947b0-c0a5-4cfd-9d06-b64e28687cb0 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:33.755256 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 400/812] 10.222.0.39 () {56 vars in 1124 bytes} [Fri Aug 5 19:50:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:33.756034 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d947b0-c0a5-4cfd-9d06-b64e28687cb0 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:33.779903 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a315460-3dd2-47b7-ab79-11b555dbc0d4 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:33.814223 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f85e32-e6ec-4870-bdbd-cf1d15043bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:33.814837 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f85e32-e6ec-4870-bdbd-cf1d15043bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:33.815120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f85e32-e6ec-4870-bdbd-cf1d15043bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:33.815391 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f85e32-e6ec-4870-bdbd-cf1d15043bd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:33.816076 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f85e32-e6ec-4870-bdbd-cf1d15043bd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:33.816409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f85e32-e6ec-4870-bdbd-cf1d15043bd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:33.817770 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f85e32-e6ec-4870-bdbd-cf1d15043bd8 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:33.834367 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 413/813] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:33.834367 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a688191-0528-411a-a395-594e085746e4 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:33.834367 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a688191-0528-411a-a395-594e085746e4 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:33.834367 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a688191-0528-411a-a395-594e085746e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:33.849162 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-8a688191-0528-411a-a395-594e085746e4 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:33.850382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-8a688191-0528-411a-a395-594e085746e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:33.862485 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d947b0-c0a5-4cfd-9d06-b64e28687cb0 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:33.892898 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:33.899253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b4e4d-a594-47ae-9cdb-0995c3fdc785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:33.899873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b4e4d-a594-47ae-9cdb-0995c3fdc785 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:33.900158 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b4e4d-a594-47ae-9cdb-0995c3fdc785 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:33.900440 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b4e4d-a594-47ae-9cdb-0995c3fdc785 None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:33.901161 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b4e4d-a594-47ae-9cdb-0995c3fdc785 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:33.901478 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b4e4d-a594-47ae-9cdb-0995c3fdc785 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:33.902973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b4e4d-a594-47ae-9cdb-0995c3fdc785 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:33.914849 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 401/814] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:33.915612 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b4e4d-a594-47ae-9cdb-0995c3fdc785 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:33.918906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 414/815] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:33.919934 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a688191-0528-411a-a395-594e085746e4 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:33.923386 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:34.022406 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a688191-0528-411a-a395-594e085746e4 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:34.030650 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b4e4d-a594-47ae-9cdb-0995c3fdc785 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:34.056753 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cec85d-f229-4795-8b5b-07edc59809db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:34.057371 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cec85d-f229-4795-8b5b-07edc59809db None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:34.057649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cec85d-f229-4795-8b5b-07edc59809db None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:34.058523 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cec85d-f229-4795-8b5b-07edc59809db None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:34.059194 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cec85d-f229-4795-8b5b-07edc59809db None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:34.059500 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cec85d-f229-4795-8b5b-07edc59809db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:34.060887 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cec85d-f229-4795-8b5b-07edc59809db None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:34.065700 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 415/816] 10.222.0.39 () {56 vars in 1124 bytes} [Fri Aug 5 19:50:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:34.066466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cec85d-f229-4795-8b5b-07edc59809db None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:34.071536 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c6e0d-0482-4922-8b41-4ef1be445c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:34.072203 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c6e0d-0482-4922-8b41-4ef1be445c66 None admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:34.072482 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c6e0d-0482-4922-8b41-4ef1be445c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:34.072794 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c6e0d-0482-4922-8b41-4ef1be445c66 None admin] PATH_INFO: `/v3/projects/80379e2104644e98a2e85d4e4af1b365/users/91d3d43cb0bd4ca3b8e8f10e29cc421c/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:34.075279 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c6e0d-0482-4922-8b41-4ef1be445c66 None admin] RBAC: Authorizing `identity:create_grant(project_id=80379e2104644e98a2e85d4e4af1b365, user_id=91d3d43cb0bd4ca3b8e8f10e29cc421c, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:34.076729 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c6e0d-0482-4922-8b41-4ef1be445c66 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:34.090079 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 402/817] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Aug 5 19:50:33 2022] PUT /identity/v3/projects/80379e2104644e98a2e85d4e4af1b365/users/91d3d43cb0bd4ca3b8e8f10e29cc421c/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:34.090881 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c6e0d-0482-4922-8b41-4ef1be445c66 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:34.152319 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:34.169017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:34.196827 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cec85d-f229-4795-8b5b-07edc59809db None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:34.228133 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c6e0d-0482-4922-8b41-4ef1be445c66 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:34.242461 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d3c91-e529-4e35-aa2a-69d175217956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:34.243469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d3c91-e529-4e35-aa2a-69d175217956 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:34.243469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d3c91-e529-4e35-aa2a-69d175217956 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:34.243654 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d3c91-e529-4e35-aa2a-69d175217956 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:34.253464 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97d3c91-e529-4e35-aa2a-69d175217956 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:34.254938 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97d3c91-e529-4e35-aa2a-69d175217956 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:34.268011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-e97d3c91-e529-4e35-aa2a-69d175217956 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:50:34.268011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:34.268011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:34.268011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:34.268011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:34.268011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:34.268011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:34.268011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:34.268011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:34.268011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:34.268011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:34.268011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:34.268011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:34.268011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:34.268011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:34.268011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:34.269225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:34.269225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:34.269225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:34.269225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:34.269225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:34.269225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:34.269225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:34.269225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:34.269225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:34.269225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:34.269225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:34.269225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:34.269225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:34.269225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:34.269225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:34.269225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:34.269225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:34.269225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:34.270492 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:34.270492 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:34.270492 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:34.270492 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:34.270492 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:34.270492 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:34.270492 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:34.270492 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:34.270492 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:50:34.270492 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:34.270492 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 416/818] 10.222.0.39 () {56 vars in 1117 bytes} [Fri Aug 5 19:50:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:34.270492 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d3c91-e529-4e35-aa2a-69d175217956 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:34.277283 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f11761-f7a1-4c7e-bd5b-58a38c36587f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:34.278093 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f11761-f7a1-4c7e-bd5b-58a38c36587f None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:34.278441 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f11761-f7a1-4c7e-bd5b-58a38c36587f None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:34.278779 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f11761-f7a1-4c7e-bd5b-58a38c36587f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:34.287462 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f11761-f7a1-4c7e-bd5b-58a38c36587f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:34.289041 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f11761-f7a1-4c7e-bd5b-58a38c36587f None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:34.302166 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-37f11761-f7a1-4c7e-bd5b-58a38c36587f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:34.302166 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:34.302166 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:34.302166 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:34.302166 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:34.302166 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:34.302166 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:34.302166 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:34.302166 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:34.302166 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:34.302166 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:34.302166 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:34.302166 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:34.302166 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:34.302166 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:34.303100 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:34.303100 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:34.303100 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:34.303100 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:34.303100 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:34.303100 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:34.303100 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:34.303100 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:34.303100 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:34.303100 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:34.303100 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:34.303100 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:34.303100 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:34.303100 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:34.303100 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:34.303100 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:34.303100 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:34.304407 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:34.304407 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:34.304407 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:34.304407 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:34.304407 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:34.304407 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:34.304407 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:34.304407 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:34.304407 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 403/819] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:34.304407 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f11761-f7a1-4c7e-bd5b-58a38c36587f None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:34.389716 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d3c91-e529-4e35-aa2a-69d175217956 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:34.420641 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:34.425313 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f11761-f7a1-4c7e-bd5b-58a38c36587f None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:34.430157 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e63a9-475f-4d75-885a-1ee4809d3d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:34.430858 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e63a9-475f-4d75-885a-1ee4809d3d5c None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:34.431144 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e63a9-475f-4d75-885a-1ee4809d3d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:34.431419 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e63a9-475f-4d75-885a-1ee4809d3d5c None admin] PATH_INFO: `/v3/services/network` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:34.432042 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e63a9-475f-4d75-885a-1ee4809d3d5c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:34.434343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e63a9-475f-4d75-885a-1ee4809d3d5c None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:34.437339 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:34.444070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-473e63a9-475f-4d75-885a-1ee4809d3d5c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 19:50:34.444070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:34.444070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:34.444070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:34.444070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:34.444070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:34.444070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:34.444070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:34.444070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 19:50:34.444070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:34.444070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:34.444070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:34.444070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:34.444070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:34.444070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:34.444070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:34.444070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:34.451758 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:34.451758 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:34.451758 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:34.451758 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:34.451758 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:34.451758 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:34.451758 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:34.451758 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:34.451758 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:34.451758 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:34.451758 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:34.451758 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:34.451758 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:34.451758 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:34.451758 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:34.451758 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:34.452744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:34.452744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:34.452744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:34.452744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:34.452744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:34.452744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:34.452744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:34.452744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:34.452744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:34.452744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:34.452744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:34.452744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:34.452744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 19:50:34.452744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:34.452744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 417/820] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:34 2022] GET /identity/v3/services/network => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:34.453604 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e63a9-475f-4d75-885a-1ee4809d3d5c None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:34.465641 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f061cc3-3368-4e15-8572-c46fae1c3e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:34.466416 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f061cc3-3368-4e15-8572-c46fae1c3e95 None admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:34.466700 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f061cc3-3368-4e15-8572-c46fae1c3e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:34.466979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f061cc3-3368-4e15-8572-c46fae1c3e95 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:34.467691 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f061cc3-3368-4e15-8572-c46fae1c3e95 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:34.469081 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f061cc3-3368-4e15-8572-c46fae1c3e95 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:34.496634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 404/821] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:34 2022] POST /identity/v3/endpoints => generated 346 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:34.496634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f061cc3-3368-4e15-8572-c46fae1c3e95 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:34.586120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e63a9-475f-4d75-885a-1ee4809d3d5c None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:34.634106 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c087eb-66b9-4b89-99af-d2b981ecdbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:34.634828 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c087eb-66b9-4b89-99af-d2b981ecdbff None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:34.635177 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c087eb-66b9-4b89-99af-d2b981ecdbff None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:34.635520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c087eb-66b9-4b89-99af-d2b981ecdbff None admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:34.637052 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c087eb-66b9-4b89-99af-d2b981ecdbff None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:34.637496 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c087eb-66b9-4b89-99af-d2b981ecdbff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:34.638838 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f061cc3-3368-4e15-8572-c46fae1c3e95 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:34.639368 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c087eb-66b9-4b89-99af-d2b981ecdbff None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:34.665868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 418/822] 10.222.0.39 () {56 vars in 1149 bytes} [Fri Aug 5 19:50:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:34.666994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c087eb-66b9-4b89-99af-d2b981ecdbff None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:34.698677 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99311716-5259-42d4-945f-f99003e13beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:34.699295 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99311716-5259-42d4-945f-f99003e13beb None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:34.699677 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99311716-5259-42d4-945f-f99003e13beb None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:34.699948 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99311716-5259-42d4-945f-f99003e13beb None admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:34.700731 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99311716-5259-42d4-945f-f99003e13beb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:34.701035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99311716-5259-42d4-945f-f99003e13beb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:34.702474 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99311716-5259-42d4-945f-f99003e13beb None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:34.716370 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 405/823] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:34.717335 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99311716-5259-42d4-945f-f99003e13beb None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:34.729184 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:34.751137 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:34.804608 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c087eb-66b9-4b89-99af-d2b981ecdbff None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:34.832879 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99311716-5259-42d4-945f-f99003e13beb None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:34.846996 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d8c2a-015e-412c-a70c-f7e741a39559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:34.847856 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d8c2a-015e-412c-a70c-f7e741a39559 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:34.848292 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d8c2a-015e-412c-a70c-f7e741a39559 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:34.848770 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d8c2a-015e-412c-a70c-f7e741a39559 None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:34.849613 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d8c2a-015e-412c-a70c-f7e741a39559 None admin] RBAC: Adding query filter params (name=network) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:34.850072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d8c2a-015e-412c-a70c-f7e741a39559 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:34.851588 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d8c2a-015e-412c-a70c-f7e741a39559 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:34.857225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 419/824] 10.222.0.39 () {56 vars in 1112 bytes} [Fri Aug 5 19:50:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:34.858288 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d8c2a-015e-412c-a70c-f7e741a39559 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:34.879486 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5a2f1-0706-4aa7-b1f3-a5160a786045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:34.880420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5a2f1-0706-4aa7-b1f3-a5160a786045 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:34.881087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5a2f1-0706-4aa7-b1f3-a5160a786045 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:34.881555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5a2f1-0706-4aa7-b1f3-a5160a786045 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:34.895331 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5a2f1-0706-4aa7-b1f3-a5160a786045 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:34.897646 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5a2f1-0706-4aa7-b1f3-a5160a786045 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:34.918936 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-2cf5a2f1-0706-4aa7-b1f3-a5160a786045 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:34.918936 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:34.918936 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:34.918936 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:34.918936 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:34.918936 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:34.918936 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:34.918936 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:34.918936 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:34.918936 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:34.918936 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:34.918936 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:34.918936 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:34.918936 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:34.918936 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:34.919895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:34.919895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:34.919895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:34.919895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:34.919895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:34.919895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:34.919895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:34.919895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:34.919895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:34.919895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:34.919895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:34.919895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:34.919895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:34.919895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:34.919895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:34.919895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:34.919895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:34.920828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:34.920828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:34.920828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:34.920828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:34.923350 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 406/825] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:34.924467 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5a2f1-0706-4aa7-b1f3-a5160a786045 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:34.984417 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d8c2a-015e-412c-a70c-f7e741a39559 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:34.997463 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:35.024915 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75990042-3576-4b65-b1a4-6b30ccd36a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:35.025617 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75990042-3576-4b65-b1a4-6b30ccd36a88 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:35.025999 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75990042-3576-4b65-b1a4-6b30ccd36a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:35.026398 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75990042-3576-4b65-b1a4-6b30ccd36a88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:35.028116 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75990042-3576-4b65-b1a4-6b30ccd36a88 None admin] RBAC: Adding query filter params (role.id=89919f784ad144529b07c8ab92ee7d48, scope.project.id=80379e2104644e98a2e85d4e4af1b365, user.id=95e9f085e4f04d118373b8abff27543a) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:35.028544 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75990042-3576-4b65-b1a4-6b30ccd36a88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:35.030097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75990042-3576-4b65-b1a4-6b30ccd36a88 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:35.043145 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 420/826] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:50:34 2022] GET /identity/v3/role_assignments?user.id=95e9f085e4f04d118373b8abff27543a&scope.project.id=80379e2104644e98a2e85d4e4af1b365&role.id=89919f784ad144529b07c8ab92ee7d48 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:35.043145 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75990042-3576-4b65-b1a4-6b30ccd36a88 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:35.048986 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:35.093386 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5a2f1-0706-4aa7-b1f3-a5160a786045 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:35.197621 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75990042-3576-4b65-b1a4-6b30ccd36a88 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:35.212923 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a43eab-8f20-4c32-a424-c668df1d9abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:35.212923 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a43eab-8f20-4c32-a424-c668df1d9abe None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:35.212923 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a43eab-8f20-4c32-a424-c668df1d9abe None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:35.216048 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a43eab-8f20-4c32-a424-c668df1d9abe None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:35.217373 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a43eab-8f20-4c32-a424-c668df1d9abe None admin] RBAC: Adding query filter params (type=network) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:35.217851 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a43eab-8f20-4c32-a424-c668df1d9abe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:35.219384 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a43eab-8f20-4c32-a424-c668df1d9abe None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:35.227646 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 407/827] 10.222.0.39 () {56 vars in 1112 bytes} [Fri Aug 5 19:50:34 2022] GET /identity/v3/services?type=network => generated 350 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:35.233724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a43eab-8f20-4c32-a424-c668df1d9abe None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:35.258289 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33afb44d-947e-4e9e-b4ed-09a15e0d1963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:35.259082 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33afb44d-947e-4e9e-b4ed-09a15e0d1963 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:35.259480 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33afb44d-947e-4e9e-b4ed-09a15e0d1963 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:35.259760 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33afb44d-947e-4e9e-b4ed-09a15e0d1963 None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:35.260471 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33afb44d-947e-4e9e-b4ed-09a15e0d1963 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:35.260908 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33afb44d-947e-4e9e-b4ed-09a15e0d1963 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:35.263240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33afb44d-947e-4e9e-b4ed-09a15e0d1963 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:35.281441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 421/828] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:35.285946 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33afb44d-947e-4e9e-b4ed-09a15e0d1963 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:35.336868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:35.381334 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a43eab-8f20-4c32-a424-c668df1d9abe None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:35.422867 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328080c3-91b9-47de-b855-72181cdf3385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:35.423690 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328080c3-91b9-47de-b855-72181cdf3385 None admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:35.423812 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328080c3-91b9-47de-b855-72181cdf3385 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:35.424080 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328080c3-91b9-47de-b855-72181cdf3385 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:35.424824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328080c3-91b9-47de-b855-72181cdf3385 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:35.426258 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328080c3-91b9-47de-b855-72181cdf3385 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:35.439932 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33afb44d-947e-4e9e-b4ed-09a15e0d1963 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:35.445335 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:35.455872 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 408/829] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:35 2022] POST /identity/v3/endpoints => generated 337 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:35.494240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa12a8c-4ef6-487b-814d-fc1187d1374c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:35.495083 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa12a8c-4ef6-487b-814d-fc1187d1374c None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:35.495997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa12a8c-4ef6-487b-814d-fc1187d1374c None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:35.496408 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa12a8c-4ef6-487b-814d-fc1187d1374c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:35.504558 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa12a8c-4ef6-487b-814d-fc1187d1374c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:35.506142 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa12a8c-4ef6-487b-814d-fc1187d1374c None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:35.525201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-baa12a8c-4ef6-487b-814d-fc1187d1374c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:35.525201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:35.525201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:35.525201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:35.525201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:35.525201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:35.525201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:35.525201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:35.525201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:35.525201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:35.525201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:35.525201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:35.525201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:35.525201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:35.525201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:35.526274 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:35.526274 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:35.526274 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:35.526274 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:35.526274 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:35.526274 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:35.526274 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:35.526274 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:35.526274 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:35.526274 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:35.526274 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:35.526274 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:35.526274 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:35.526274 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:35.526274 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:35.526274 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:35.526274 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:35.527310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:35.527310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:35.527310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:35.527310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:35.527310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:35.527310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:35.527310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:35.527310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:35.527310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 422/830] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:35.532545 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328080c3-91b9-47de-b855-72181cdf3385 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:35.694834 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328080c3-91b9-47de-b855-72181cdf3385 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:35.749840 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1049870a-c226-4ab0-9d30-766e2c76233a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:35.750902 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049870a-c226-4ab0-9d30-766e2c76233a None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:35.751407 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049870a-c226-4ab0-9d30-766e2c76233a None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:35.751887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049870a-c226-4ab0-9d30-766e2c76233a None admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:35.752893 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1049870a-c226-4ab0-9d30-766e2c76233a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:35.753425 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1049870a-c226-4ab0-9d30-766e2c76233a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:35.755084 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1049870a-c226-4ab0-9d30-766e2c76233a None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:35.774636 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 409/831] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:35.780727 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa12a8c-4ef6-487b-814d-fc1187d1374c None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:35.919555 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:35.930593 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa12a8c-4ef6-487b-814d-fc1187d1374c None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:35.970156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e06c3-9bf5-493a-a351-b838a77c0c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:35.970758 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e06c3-9bf5-493a-a351-b838a77c0c12 None admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:35.971042 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e06c3-9bf5-493a-a351-b838a77c0c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:35.971321 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e06c3-9bf5-493a-a351-b838a77c0c12 None admin] PATH_INFO: `/v3/projects/80379e2104644e98a2e85d4e4af1b365/users/3488a7d2258242c0859a105c1d162767/roles/7ebfd9e58fe549f08cdaec5f906a38a6` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:35.988607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e06c3-9bf5-493a-a351-b838a77c0c12 None admin] RBAC: Authorizing `identity:create_grant(project_id=80379e2104644e98a2e85d4e4af1b365, user_id=3488a7d2258242c0859a105c1d162767, role_id=7ebfd9e58fe549f08cdaec5f906a38a6)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:35.990063 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e06c3-9bf5-493a-a351-b838a77c0c12 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:36.007701 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 423/832] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Aug 5 19:50:35 2022] PUT /identity/v3/projects/80379e2104644e98a2e85d4e4af1b365/users/3488a7d2258242c0859a105c1d162767/roles/7ebfd9e58fe549f08cdaec5f906a38a6 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:50:36.330190 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ef50d-51fe-474e-9d6f-fd58fb482f70 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:36.330447 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ef50d-51fe-474e-9d6f-fd58fb482f70 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:36.330821 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ef50d-51fe-474e-9d6f-fd58fb482f70 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:36.332196 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 410/833] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:36.337816 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892b7ff-93cd-4755-910c-062b4cec8ffe None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:36.338155 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892b7ff-93cd-4755-910c-062b4cec8ffe None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:36.338405 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892b7ff-93cd-4755-910c-062b4cec8ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:36.358248 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-5892b7ff-93cd-4755-910c-062b4cec8ffe None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:36.359457 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5892b7ff-93cd-4755-910c-062b4cec8ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:36.431967 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 424/834] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:36.455718 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ef50d-51fe-474e-9d6f-fd58fb482f70 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:36.521799 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:36.569972 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ef50d-51fe-474e-9d6f-fd58fb482f70 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:36.609759 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3650f-ac92-44c2-b0b3-c5c5e8dcf7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:36.610935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3650f-ac92-44c2-b0b3-c5c5e8dcf7cf None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:36.611424 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3650f-ac92-44c2-b0b3-c5c5e8dcf7cf None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:36.611897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3650f-ac92-44c2-b0b3-c5c5e8dcf7cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:36.624392 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e3650f-ac92-44c2-b0b3-c5c5e8dcf7cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:36.629825 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-33e3650f-ac92-44c2-b0b3-c5c5e8dcf7cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:50:36.629825 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:36.629825 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:36.629825 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:36.629825 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:36.629825 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:36.629825 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:36.629825 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:36.629825 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:36.629825 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:36.629825 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:36.629825 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:36.629825 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:36.629825 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:36.629825 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:36.629825 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:36.631185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:36.631185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:36.631185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:36.631185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:36.631185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:36.631185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:36.631185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:36.631185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:36.631185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:36.631185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:36.631185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:36.631185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:36.631185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:36.631185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:36.631185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:36.631185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:36.632359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:36.632359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:36.632359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:36.632359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:36.632359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:36.632359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:36.632359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:36.632359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:36.632359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:36.632359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:36.632359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:50:36.632359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:36.635347 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 411/835] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:50:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:36.641479 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5892b7ff-93cd-4755-910c-062b4cec8ffe None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:36.716636 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:36.752027 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5892b7ff-93cd-4755-910c-062b4cec8ffe None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:36.766501 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54714beb-5991-495c-9cee-5248a8a600e9 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:36.766817 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54714beb-5991-495c-9cee-5248a8a600e9 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:36.767047 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54714beb-5991-495c-9cee-5248a8a600e9 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:36.768528 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 412/836] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:36.774862 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f60d52-3578-40fc-95f0-36c1bd624dfa None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:36.775144 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f60d52-3578-40fc-95f0-36c1bd624dfa None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:36.775420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f60d52-3578-40fc-95f0-36c1bd624dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:36.794600 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-70f60d52-3578-40fc-95f0-36c1bd624dfa None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:36.795777 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-70f60d52-3578-40fc-95f0-36c1bd624dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:36.799263 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07cfa5f-db9e-4368-bd30-6370a82b93d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:36.800188 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cfa5f-db9e-4368-bd30-6370a82b93d2 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:36.800732 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cfa5f-db9e-4368-bd30-6370a82b93d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:36.801205 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cfa5f-db9e-4368-bd30-6370a82b93d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:36.802136 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07cfa5f-db9e-4368-bd30-6370a82b93d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:36.802876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07cfa5f-db9e-4368-bd30-6370a82b93d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:36.804488 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07cfa5f-db9e-4368-bd30-6370a82b93d2 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:36.815061 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 425/837] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:50:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:36.821768 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07cfa5f-db9e-4368-bd30-6370a82b93d2 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:36.900601 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 413/838] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:50:36.918926 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f60d52-3578-40fc-95f0-36c1bd624dfa None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:36.954023 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07cfa5f-db9e-4368-bd30-6370a82b93d2 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:36.983226 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:36.988603 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:36.996188 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c497e-bdab-46a1-a7eb-640d96604a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:36.997287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c497e-bdab-46a1-a7eb-640d96604a46 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:36.997771 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c497e-bdab-46a1-a7eb-640d96604a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:36.998270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c497e-bdab-46a1-a7eb-640d96604a46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:37.010435 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c497e-bdab-46a1-a7eb-640d96604a46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:37.013269 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c497e-bdab-46a1-a7eb-640d96604a46 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:37.032718 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-486c497e-bdab-46a1-a7eb-640d96604a46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:37.032718 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:37.032718 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:37.032718 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:37.032718 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:37.032718 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:37.032718 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:37.032718 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:37.032718 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:37.032718 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:37.032718 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:37.032718 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:37.032718 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:37.032718 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:37.032718 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:37.033691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:37.033691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:37.033691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:37.033691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:37.033691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:37.033691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:37.033691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:37.033691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:37.033691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:37.033691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:37.033691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:37.033691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:37.033691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:37.033691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:37.033691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:37.033691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:37.033691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:37.034611 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:37.034611 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:37.034611 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:37.034611 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:37.036935 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 426/839] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:37.043182 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c497e-bdab-46a1-a7eb-640d96604a46 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:37.056984 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f60d52-3578-40fc-95f0-36c1bd624dfa None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:37.095669 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d8e53-1c8a-477b-9bf9-3d2a6db3a153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:37.095915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d8e53-1c8a-477b-9bf9-3d2a6db3a153 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:37.096151 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d8e53-1c8a-477b-9bf9-3d2a6db3a153 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:37.096432 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d8e53-1c8a-477b-9bf9-3d2a6db3a153 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:37.097113 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d8e53-1c8a-477b-9bf9-3d2a6db3a153 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:37.098603 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d8e53-1c8a-477b-9bf9-3d2a6db3a153 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:37.111500 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-309d8e53-1c8a-477b-9bf9-3d2a6db3a153 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 19:50:37.111500 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:37.111500 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:37.111500 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:37.111500 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:37.111500 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:37.111500 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:37.111500 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:37.111500 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 19:50:37.111500 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:37.111500 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:37.111500 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:37.111500 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:37.111500 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:37.111500 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:37.111500 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:37.111500 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:37.112925 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:37.112925 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:37.112925 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:37.112925 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:37.112925 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:37.112925 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:37.112925 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:37.112925 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:37.112925 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:37.112925 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:37.112925 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:37.112925 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:37.112925 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:37.112925 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:37.112925 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:37.112925 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:37.114288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:37.114288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:37.114288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:37.114288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:37.114288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:37.114288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:37.114288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:37.114288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:37.114288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:37.114288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:37.114288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:37.114288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:37.114288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 19:50:37.114288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:37.117665 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 414/840] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Aug 5 19:50:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:37.118610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d8e53-1c8a-477b-9bf9-3d2a6db3a153 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:37.222058 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c497e-bdab-46a1-a7eb-640d96604a46 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:37.266199 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d8e53-1c8a-477b-9bf9-3d2a6db3a153 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:37.283907 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:37.294468 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3059e8-7a81-4db0-bbd7-8366f8102b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:37.295601 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3059e8-7a81-4db0-bbd7-8366f8102b42 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:37.295919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3059e8-7a81-4db0-bbd7-8366f8102b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:37.296291 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3059e8-7a81-4db0-bbd7-8366f8102b42 None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:37.297723 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3059e8-7a81-4db0-bbd7-8366f8102b42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:37.298268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3059e8-7a81-4db0-bbd7-8366f8102b42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:37.301634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3059e8-7a81-4db0-bbd7-8366f8102b42 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:37.314907 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f606e-27d3-4a88-ab30-ad5ee7c2d335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:37.315903 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f606e-27d3-4a88-ab30-ad5ee7c2d335 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:37.316364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f606e-27d3-4a88-ab30-ad5ee7c2d335 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:37.316858 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f606e-27d3-4a88-ab30-ad5ee7c2d335 None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:37.317760 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835f606e-27d3-4a88-ab30-ad5ee7c2d335 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:37.318298 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835f606e-27d3-4a88-ab30-ad5ee7c2d335 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:37.319959 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835f606e-27d3-4a88-ab30-ad5ee7c2d335 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:37.323764 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 427/841] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:37.326077 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 415/842] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Aug 5 19:50:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:37.329727 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3059e8-7a81-4db0-bbd7-8366f8102b42 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:37.331774 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f606e-27d3-4a88-ab30-ad5ee7c2d335 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:37.391353 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:37.461983 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3059e8-7a81-4db0-bbd7-8366f8102b42 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:37.473269 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f606e-27d3-4a88-ab30-ad5ee7c2d335 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:37.508646 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231582d0-3c39-496d-b820-c266939dfa21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:37.509580 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231582d0-3c39-496d-b820-c266939dfa21 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:37.509870 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231582d0-3c39-496d-b820-c266939dfa21 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:37.510147 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231582d0-3c39-496d-b820-c266939dfa21 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:37.519170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd9b5d-2060-4307-8cd4-6beb4bb04c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:37.519405 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231582d0-3c39-496d-b820-c266939dfa21 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:37.520402 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd9b5d-2060-4307-8cd4-6beb4bb04c54 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:37.520867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231582d0-3c39-496d-b820-c266939dfa21 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:37.521222 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd9b5d-2060-4307-8cd4-6beb4bb04c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:37.521739 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd9b5d-2060-4307-8cd4-6beb4bb04c54 None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:37.522786 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fd9b5d-2060-4307-8cd4-6beb4bb04c54 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:37.523598 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fd9b5d-2060-4307-8cd4-6beb4bb04c54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:37.525469 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fd9b5d-2060-4307-8cd4-6beb4bb04c54 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:37.533381 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 416/843] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Aug 5 19:50:37 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:37.539213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-231582d0-3c39-496d-b820-c266939dfa21 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:50:37.539213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:37.539213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:37.539213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:37.539213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:37.539213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:37.539213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:37.539213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:37.539213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:37.539213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:37.539213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:37.539213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:37.539213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:37.539213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:37.539213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:37.539213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:37.540241 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:37.540241 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:37.540241 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:37.540241 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:37.540241 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:37.540241 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:37.540241 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:37.540241 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:37.540241 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:37.540241 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:37.540241 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:37.540241 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:37.540241 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:37.540241 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:37.540241 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:37.540241 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:37.540241 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:37.540241 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:37.541324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:37.541324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:37.541324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:37.541324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:37.541324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:37.541324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:37.541324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:37.541324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:37.541324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:50:37.541324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:37.544522 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 428/844] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:50:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:37.552652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd9b5d-2060-4307-8cd4-6beb4bb04c54 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:37.606453 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:37.686057 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd9b5d-2060-4307-8cd4-6beb4bb04c54 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:37.726518 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76887c4d-d422-49da-9c4d-e02aa42d5a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:37.727066 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76887c4d-d422-49da-9c4d-e02aa42d5a8d None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:37.727348 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76887c4d-d422-49da-9c4d-e02aa42d5a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:37.727624 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76887c4d-d422-49da-9c4d-e02aa42d5a8d None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:37.728428 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76887c4d-d422-49da-9c4d-e02aa42d5a8d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:37.728827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76887c4d-d422-49da-9c4d-e02aa42d5a8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:37.730993 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76887c4d-d422-49da-9c4d-e02aa42d5a8d None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:37.743441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6d51d-6999-4016-a295-116760081899 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:37.744060 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6d51d-6999-4016-a295-116760081899 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:37.744060 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6d51d-6999-4016-a295-116760081899 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:37.746291 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 429/845] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:50:37.751759 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a9920-bb50-4627-82d6-ceebd55be8b7 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:37.752050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a9920-bb50-4627-82d6-ceebd55be8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:37.752329 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a9920-bb50-4627-82d6-ceebd55be8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:37.752512 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 417/846] 10.222.0.39 () {56 vars in 1135 bytes} [Fri Aug 5 19:50:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:37.758203 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76887c4d-d422-49da-9c4d-e02aa42d5a8d None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:37.785197 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-224a9920-bb50-4627-82d6-ceebd55be8b7 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:37.786935 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-224a9920-bb50-4627-82d6-ceebd55be8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:37.867682 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 430/847] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:37 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:37.882471 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:37.886792 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76887c4d-d422-49da-9c4d-e02aa42d5a8d None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:37.892263 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a9920-bb50-4627-82d6-ceebd55be8b7 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:37.942352 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:37.950827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c7c2a5-1b33-414b-8a6c-f506666059d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:37.951460 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7c2a5-1b33-414b-8a6c-f506666059d8 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:37.951739 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7c2a5-1b33-414b-8a6c-f506666059d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:37.952008 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7c2a5-1b33-414b-8a6c-f506666059d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:38.032524 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a9920-bb50-4627-82d6-ceebd55be8b7 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:38.060032 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c7c2a5-1b33-414b-8a6c-f506666059d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:38.061502 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c7c2a5-1b33-414b-8a6c-f506666059d8 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:38.094706 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d371b9-8cf9-4a87-9c8a-4e2283483667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:38.095876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d371b9-8cf9-4a87-9c8a-4e2283483667 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:38.096401 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d371b9-8cf9-4a87-9c8a-4e2283483667 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:38.096955 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d371b9-8cf9-4a87-9c8a-4e2283483667 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:38.097825 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d371b9-8cf9-4a87-9c8a-4e2283483667 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:38.099647 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d371b9-8cf9-4a87-9c8a-4e2283483667 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:38.110655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-28d371b9-8cf9-4a87-9c8a-4e2283483667 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 19:50:38.110655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:38.110655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:38.110655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:38.110655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:38.110655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:38.110655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:38.110655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:38.110655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 19:50:38.110655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:38.110655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:38.110655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:38.110655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:38.110655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:38.110655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:38.110655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:38.112094 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:38.112094 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:38.112094 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:38.112094 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:38.112094 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:38.112094 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:38.112094 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:38.112094 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:38.112094 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:38.112094 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:38.112094 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:38.112094 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:38.112094 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:38.112094 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:38.112094 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:38.112094 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:38.113596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:38.113596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:38.113596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:38.113596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:38.113596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:38.113596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:38.113596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:38.113596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:38.113596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:38.113596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:38.113596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:38.113596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:38.113596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:38.113596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 19:50:38.113596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:38.125211 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 431/848] 10.222.0.39 () {56 vars in 1132 bytes} [Fri Aug 5 19:50:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:38.130237 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d371b9-8cf9-4a87-9c8a-4e2283483667 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:38.186318 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:38.195889 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 418/849] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:50:37 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 19:50:38.198114 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c7c2a5-1b33-414b-8a6c-f506666059d8 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:38.274190 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:38.278827 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d371b9-8cf9-4a87-9c8a-4e2283483667 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:38.317074 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f00234-d668-4c04-8749-4689224a9a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:38.317720 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f00234-d668-4c04-8749-4689224a9a0a None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:38.317997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f00234-d668-4c04-8749-4689224a9a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:38.318264 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f00234-d668-4c04-8749-4689224a9a0a None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:38.318938 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f00234-d668-4c04-8749-4689224a9a0a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:38.319238 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f00234-d668-4c04-8749-4689224a9a0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:38.320332 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c7c2a5-1b33-414b-8a6c-f506666059d8 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:38.320633 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f00234-d668-4c04-8749-4689224a9a0a None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:38.331310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 432/850] 10.222.0.39 () {56 vars in 1126 bytes} [Fri Aug 5 19:50:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:38.331310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e01b08-3513-4934-9720-22980100f5c0 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:38.331310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e01b08-3513-4934-9720-22980100f5c0 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:38.331310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e01b08-3513-4934-9720-22980100f5c0 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:38.331310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 433/851] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:38.334833 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e01b08-3513-4934-9720-22980100f5c0 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:38.363716 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5fc253-028d-4f56-9643-e15fd0af3735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:38.364469 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fc253-028d-4f56-9643-e15fd0af3735 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:38.364779 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fc253-028d-4f56-9643-e15fd0af3735 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:38.365042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fc253-028d-4f56-9643-e15fd0af3735 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:38.377761 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5fc253-028d-4f56-9643-e15fd0af3735 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:38.379214 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5fc253-028d-4f56-9643-e15fd0af3735 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:38.395915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-5d5fc253-028d-4f56-9643-e15fd0af3735 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:38.395915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:38.395915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:38.395915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:38.395915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:38.395915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:38.395915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:38.395915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:38.395915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:38.395915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:38.395915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:38.395915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:38.395915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:38.395915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:38.395915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:38.397364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:38.397364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:38.397364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:38.397364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:38.397364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:38.397364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:38.397364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:38.397364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:38.397364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:38.397364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:38.397364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:38.397364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:38.397364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:38.397364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:38.397364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:38.397364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:38.397364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:38.398905 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:38.398905 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:38.398905 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:38.398905 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:38.398905 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 419/852] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:38.399188 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f4c11-ed5c-4ebb-87fb-e96912537c7c None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:38.399256 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f4c11-ed5c-4ebb-87fb-e96912537c7c None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:38.399525 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f4c11-ed5c-4ebb-87fb-e96912537c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:38.418372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-622f4c11-ed5c-4ebb-87fb-e96912537c7c None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:38.419581 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-622f4c11-ed5c-4ebb-87fb-e96912537c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:38.452790 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e01b08-3513-4934-9720-22980100f5c0 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:38.470054 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:38.489967 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba633677-f0fc-483e-8fc7-0858ca29457f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:38.490579 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba633677-f0fc-483e-8fc7-0858ca29457f None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:38.490861 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba633677-f0fc-483e-8fc7-0858ca29457f None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:38.491132 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba633677-f0fc-483e-8fc7-0858ca29457f None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:38.491792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba633677-f0fc-483e-8fc7-0858ca29457f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:38.492085 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba633677-f0fc-483e-8fc7-0858ca29457f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:38.500535 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 420/853] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:38.500535 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f4c11-ed5c-4ebb-87fb-e96912537c7c None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:38.501221 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba633677-f0fc-483e-8fc7-0858ca29457f None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:38.501221 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 434/854] 10.222.0.39 () {56 vars in 1126 bytes} [Fri Aug 5 19:50:38 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:38.520442 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba633677-f0fc-483e-8fc7-0858ca29457f None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:38.569378 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:38.644682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f4c11-ed5c-4ebb-87fb-e96912537c7c None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:38.689654 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba633677-f0fc-483e-8fc7-0858ca29457f None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:38.692098 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e190ca-2d7e-4975-983b-1d9899bd9f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:38.693420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e190ca-2d7e-4975-983b-1d9899bd9f14 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:38.693920 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e190ca-2d7e-4975-983b-1d9899bd9f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:38.695186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e190ca-2d7e-4975-983b-1d9899bd9f14 None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:38.695700 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e190ca-2d7e-4975-983b-1d9899bd9f14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:38.696815 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e190ca-2d7e-4975-983b-1d9899bd9f14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:38.698871 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e190ca-2d7e-4975-983b-1d9899bd9f14 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:38.716308 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 421/855] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:38.726094 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e190ca-2d7e-4975-983b-1d9899bd9f14 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:38.752029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499062ac-480d-4129-9f5f-9ba5a6d1e9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:38.753090 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499062ac-480d-4129-9f5f-9ba5a6d1e9ac None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:38.753537 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499062ac-480d-4129-9f5f-9ba5a6d1e9ac None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:38.753989 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499062ac-480d-4129-9f5f-9ba5a6d1e9ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:38.773042 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499062ac-480d-4129-9f5f-9ba5a6d1e9ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:38.784323 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-499062ac-480d-4129-9f5f-9ba5a6d1e9ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:38.784323 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:38.784323 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:38.784323 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:38.784323 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:38.784323 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:38.784323 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:38.784323 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:38.784323 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:38.784323 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:38.784323 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:38.784323 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:38.784323 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:38.784323 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:38.784323 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:38.784323 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:38.785355 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:38.785355 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:38.785355 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:38.785355 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:38.785355 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:38.785355 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:38.785355 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:38.785355 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:38.785355 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:38.785355 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:38.785355 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:38.785355 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:38.785355 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:38.785355 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:38.785355 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:38.785355 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:38.786185 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:38.786185 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:38.786185 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:38.786185 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:38.786185 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:38.786185 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:38.786185 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:38.786185 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:38.786185 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:38.786185 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:38.786185 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:38.786185 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:38.786838 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 435/856] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Aug 5 19:50:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:38.797256 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499062ac-480d-4129-9f5f-9ba5a6d1e9ac None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:38.803961 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:38.903511 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e190ca-2d7e-4975-983b-1d9899bd9f14 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:38.952437 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:38.966367 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f4d02-6d0e-413e-ba36-0ba5839b7d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:38.967514 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f4d02-6d0e-413e-ba36-0ba5839b7d31 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:38.968006 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f4d02-6d0e-413e-ba36-0ba5839b7d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:38.968454 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f4d02-6d0e-413e-ba36-0ba5839b7d31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:38.977851 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499062ac-480d-4129-9f5f-9ba5a6d1e9ac None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:38.982472 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78f4d02-6d0e-413e-ba36-0ba5839b7d31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:38.984989 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78f4d02-6d0e-413e-ba36-0ba5839b7d31 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:39.007200 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-b78f4d02-6d0e-413e-ba36-0ba5839b7d31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:39.007200 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:39.007200 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:39.007200 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:39.007200 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:39.007200 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:39.007200 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:39.007200 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:39.007200 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:39.007200 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:39.007200 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:39.007200 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:39.007200 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:39.007200 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:39.007200 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:39.008301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:39.008301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:39.008301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:39.008301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:39.008301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:39.008301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:39.008301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:39.008301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:39.008301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:39.008301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:39.008301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:39.008301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:39.008301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:39.008301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:39.008301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:39.008301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:39.008301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:39.009336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:39.009336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:39.009336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:39.009336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:39.009336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:39.009336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:39.009336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:39.009336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:39.012635 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 422/857] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:39.016133 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f4d02-6d0e-413e-ba36-0ba5839b7d31 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:39.034794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75606651-5d68-4f19-ae09-0cd609faeb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:39.035747 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75606651-5d68-4f19-ae09-0cd609faeb48 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:39.036219 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75606651-5d68-4f19-ae09-0cd609faeb48 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:39.036721 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75606651-5d68-4f19-ae09-0cd609faeb48 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:39.037639 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75606651-5d68-4f19-ae09-0cd609faeb48 None admin] RBAC: Adding query filter params (name=service) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:39.038121 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75606651-5d68-4f19-ae09-0cd609faeb48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:39.039672 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75606651-5d68-4f19-ae09-0cd609faeb48 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:39.049385 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 436/858] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Aug 5 19:50:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:39.054863 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75606651-5d68-4f19-ae09-0cd609faeb48 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:39.151952 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:39.156104 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f4d02-6d0e-413e-ba36-0ba5839b7d31 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:39.198369 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb7f0fb-68b6-4caf-ba96-e550925d461d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:39.199331 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7f0fb-68b6-4caf-ba96-e550925d461d None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:39.199786 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7f0fb-68b6-4caf-ba96-e550925d461d None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:39.200232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7f0fb-68b6-4caf-ba96-e550925d461d None admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:39.201251 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb7f0fb-68b6-4caf-ba96-e550925d461d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:39.201776 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb7f0fb-68b6-4caf-ba96-e550925d461d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:39.202970 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75606651-5d68-4f19-ae09-0cd609faeb48 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:39.203609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb7f0fb-68b6-4caf-ba96-e550925d461d None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:39.224798 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 423/859] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:39.226047 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb7f0fb-68b6-4caf-ba96-e550925d461d None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:39.246889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177146c2-32f1-434f-a063-223497ad2b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:39.247897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177146c2-32f1-434f-a063-223497ad2b24 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:39.248271 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177146c2-32f1-434f-a063-223497ad2b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:39.248637 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177146c2-32f1-434f-a063-223497ad2b24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:39.264666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177146c2-32f1-434f-a063-223497ad2b24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:39.266237 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177146c2-32f1-434f-a063-223497ad2b24 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:39.279023 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:39.289487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-177146c2-32f1-434f-a063-223497ad2b24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:39.289487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:39.289487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:39.289487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:39.289487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:39.289487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:39.289487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:39.289487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:39.289487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:39.289487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:39.289487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:39.289487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:39.289487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:39.289487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:39.289487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:39.290426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:39.290426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:39.290426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:39.290426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:39.290426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:39.290426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:39.290426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:39.290426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:39.290426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:39.290426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:39.290426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:39.290426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:39.290426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:39.290426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:39.290426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:39.290426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:39.290426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:39.291309 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:39.291309 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:39.291309 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:39.291309 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:39.291309 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 437/860] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:39.297580 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177146c2-32f1-434f-a063-223497ad2b24 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:39.351142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb7f0fb-68b6-4caf-ba96-e550925d461d None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:39.391941 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2697459-6a9a-451d-a8c7-3e2711809f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:39.393202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2697459-6a9a-451d-a8c7-3e2711809f5a None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:39.393660 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2697459-6a9a-451d-a8c7-3e2711809f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:39.394140 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2697459-6a9a-451d-a8c7-3e2711809f5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:39.395375 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2697459-6a9a-451d-a8c7-3e2711809f5a None admin] RBAC: Adding query filter params (role.id=89919f784ad144529b07c8ab92ee7d48, scope.project.id=80379e2104644e98a2e85d4e4af1b365, user.id=91d3d43cb0bd4ca3b8e8f10e29cc421c) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:39.395914 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2697459-6a9a-451d-a8c7-3e2711809f5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:39.398722 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2697459-6a9a-451d-a8c7-3e2711809f5a None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:39.413170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 424/861] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:50:39 2022] GET /identity/v3/role_assignments?user.id=91d3d43cb0bd4ca3b8e8f10e29cc421c&scope.project.id=80379e2104644e98a2e85d4e4af1b365&role.id=89919f784ad144529b07c8ab92ee7d48 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:39.446234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177146c2-32f1-434f-a063-223497ad2b24 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:39.488418 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9be37-6bb6-4179-b5dd-7006ded5a2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:39.489093 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9be37-6bb6-4179-b5dd-7006ded5a2e3 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:39.489374 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9be37-6bb6-4179-b5dd-7006ded5a2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:39.489644 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9be37-6bb6-4179-b5dd-7006ded5a2e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:39.490347 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae9be37-6bb6-4179-b5dd-7006ded5a2e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:39.490651 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae9be37-6bb6-4179-b5dd-7006ded5a2e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:39.492061 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae9be37-6bb6-4179-b5dd-7006ded5a2e3 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:39.506879 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 438/862] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:39.512805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2697459-6a9a-451d-a8c7-3e2711809f5a None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:39.547405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:39.641404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2697459-6a9a-451d-a8c7-3e2711809f5a None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:39.682632 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d6784-b198-480e-9c63-be6c6043168e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:39.683216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d6784-b198-480e-9c63-be6c6043168e None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:39.683598 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d6784-b198-480e-9c63-be6c6043168e None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:39.683871 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d6784-b198-480e-9c63-be6c6043168e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:39.692258 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d6784-b198-480e-9c63-be6c6043168e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:39.693993 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d6784-b198-480e-9c63-be6c6043168e None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:39.707361 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-ae8d6784-b198-480e-9c63-be6c6043168e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:50:39.707361 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:39.707361 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:39.707361 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:39.707361 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:39.707361 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:39.707361 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:39.707361 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:39.707361 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:39.707361 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:39.707361 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:39.707361 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:39.707361 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:39.707361 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:39.707361 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:39.707361 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:39.708400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:39.708400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:39.708400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:39.708400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:39.708400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:39.708400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:39.708400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:39.708400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:39.708400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:39.708400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:39.708400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:39.708400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:39.708400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:39.708400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:39.708400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:39.708400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:39.708400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:39.708400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:39.709378 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:39.709378 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:39.709378 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:39.709378 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:39.709378 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:39.709378 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:39.709378 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:39.709378 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:39.709378 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:50:39.709378 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:39.709929 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 425/863] 10.222.0.39 () {56 vars in 1117 bytes} [Fri Aug 5 19:50:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:39.710739 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca19504-e4e7-4deb-8cd5-bdfb6fd1ca1c None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:39.711031 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca19504-e4e7-4deb-8cd5-bdfb6fd1ca1c None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:39.711312 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca19504-e4e7-4deb-8cd5-bdfb6fd1ca1c None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:39.712836 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 439/864] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:50:39.715894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d6784-b198-480e-9c63-be6c6043168e None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:39.718705 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd420e-4855-4d3c-8d44-73b4bc94d5ed None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:39.718997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd420e-4855-4d3c-8d44-73b4bc94d5ed None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:39.719347 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd420e-4855-4d3c-8d44-73b4bc94d5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:39.739499 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-07bd420e-4855-4d3c-8d44-73b4bc94d5ed None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:39.741034 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-07bd420e-4855-4d3c-8d44-73b4bc94d5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:39.808228 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:39.827305 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 440/865] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:50:39.844085 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:39.848062 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bd420e-4855-4d3c-8d44-73b4bc94d5ed None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:39.870640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d6784-b198-480e-9c63-be6c6043168e None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:39.931820 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc144d5-fb37-4e45-8ead-935ccef7f485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:39.933115 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc144d5-fb37-4e45-8ead-935ccef7f485 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:39.933351 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc144d5-fb37-4e45-8ead-935ccef7f485 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:39.933604 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc144d5-fb37-4e45-8ead-935ccef7f485 None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:39.934514 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc144d5-fb37-4e45-8ead-935ccef7f485 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:39.934827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc144d5-fb37-4e45-8ead-935ccef7f485 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:39.936508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc144d5-fb37-4e45-8ead-935ccef7f485 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:39.960219 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 426/866] 10.222.0.39 () {56 vars in 1149 bytes} [Fri Aug 5 19:50:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:39.968602 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc144d5-fb37-4e45-8ead-935ccef7f485 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:39.990881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bd420e-4855-4d3c-8d44-73b4bc94d5ed None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:40.031417 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b3aa05-894f-499e-b46b-f102d4f37406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:40.032047 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3aa05-894f-499e-b46b-f102d4f37406 None admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:40.032330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3aa05-894f-499e-b46b-f102d4f37406 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:40.032660 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3aa05-894f-499e-b46b-f102d4f37406 None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:40.033379 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b3aa05-894f-499e-b46b-f102d4f37406 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:40.034765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b3aa05-894f-499e-b46b-f102d4f37406 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:40.055493 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 441/867] 10.222.0.39 () {60 vars in 1141 bytes} [Fri Aug 5 19:50:39 2022] POST /identity/v3/services => generated 251 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:40.119606 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc144d5-fb37-4e45-8ead-935ccef7f485 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:40.173915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4afa3-c33b-46d9-b806-df64c884070c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:40.174860 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4afa3-c33b-46d9-b806-df64c884070c None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:40.175292 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4afa3-c33b-46d9-b806-df64c884070c None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:40.175712 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4afa3-c33b-46d9-b806-df64c884070c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:40.189823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:40.268042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed4afa3-c33b-46d9-b806-df64c884070c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:40.270541 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed4afa3-c33b-46d9-b806-df64c884070c None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:40.389214 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 427/868] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:50:39 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:50:40.394764 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b3aa05-894f-499e-b46b-f102d4f37406 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:40.454762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:40.523777 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b3aa05-894f-499e-b46b-f102d4f37406 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:40.569418 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513cc170-b6eb-41d7-b49c-7f034ff8d27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:40.570069 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cc170-b6eb-41d7-b49c-7f034ff8d27c None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:40.570350 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cc170-b6eb-41d7-b49c-7f034ff8d27c None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:40.570623 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cc170-b6eb-41d7-b49c-7f034ff8d27c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:40.584419 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513cc170-b6eb-41d7-b49c-7f034ff8d27c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:40.585928 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513cc170-b6eb-41d7-b49c-7f034ff8d27c None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:40.603596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-513cc170-b6eb-41d7-b49c-7f034ff8d27c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:40.603596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:40.603596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:40.603596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:40.603596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:40.603596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:40.603596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:40.603596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:40.603596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:40.603596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:40.603596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:40.603596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:40.603596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:40.603596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:40.603596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:40.605384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:40.605384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:40.605384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:40.605384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:40.605384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:40.605384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:40.605384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:40.605384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:40.605384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:40.605384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:40.605384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:40.605384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:40.605384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:40.605384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:40.605384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:40.605384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:40.605384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:40.606411 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:40.606411 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:40.606411 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:40.606411 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:40.606411 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 442/869] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:40.613713 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4afa3-c33b-46d9-b806-df64c884070c None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:40.657892 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:40.723648 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4afa3-c33b-46d9-b806-df64c884070c None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:40.763802 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67999346-5eed-44c1-87dc-2ff19e2171b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:40.764420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67999346-5eed-44c1-87dc-2ff19e2171b6 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:40.764788 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67999346-5eed-44c1-87dc-2ff19e2171b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:40.765062 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67999346-5eed-44c1-87dc-2ff19e2171b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:40.765752 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67999346-5eed-44c1-87dc-2ff19e2171b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:40.766056 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67999346-5eed-44c1-87dc-2ff19e2171b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:40.767414 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67999346-5eed-44c1-87dc-2ff19e2171b6 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:40.781182 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 428/870] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:40.786895 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513cc170-b6eb-41d7-b49c-7f034ff8d27c None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:40.846642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd922d4a-d238-41fa-8584-7cf2581d4b0e None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:40.846966 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd922d4a-d238-41fa-8584-7cf2581d4b0e None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:40.847250 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd922d4a-d238-41fa-8584-7cf2581d4b0e None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:40.848886 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 429/871] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:40.853876 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296afab-fc2d-4d12-9beb-16e2375f5ccc None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:40.854761 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296afab-fc2d-4d12-9beb-16e2375f5ccc None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:40.855048 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296afab-fc2d-4d12-9beb-16e2375f5ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:40.874340 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-7296afab-fc2d-4d12-9beb-16e2375f5ccc None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:40.876022 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-7296afab-fc2d-4d12-9beb-16e2375f5ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:40.904812 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513cc170-b6eb-41d7-b49c-7f034ff8d27c None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:40.906702 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:40.945563 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173808c0-ddc7-4732-bc86-327b6aef82f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:40.946510 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173808c0-ddc7-4732-bc86-327b6aef82f1 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:40.947189 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173808c0-ddc7-4732-bc86-327b6aef82f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:40.947477 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173808c0-ddc7-4732-bc86-327b6aef82f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:40.958488 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173808c0-ddc7-4732-bc86-327b6aef82f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:40.959624 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173808c0-ddc7-4732-bc86-327b6aef82f1 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:40.976679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 430/872] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:40 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:50:40.977577 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-173808c0-ddc7-4732-bc86-327b6aef82f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:40.977577 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:40.977577 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:40.977577 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:40.977577 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:40.977577 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:40.977577 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:40.977577 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:40.977577 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:40.977577 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:40.977577 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:40.977577 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:40.977577 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:40.977577 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:40.977577 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:40.978839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:40.978839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:40.978839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:40.978839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:40.978839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:40.978839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:40.978839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:40.978839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:40.978839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:40.978839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:40.978839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:40.978839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:40.978839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:40.978839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:40.978839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:40.978839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:40.978839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:40.979770 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:40.979770 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:40.979770 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:40.979770 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:40.979770 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:40.979770 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:40.979770 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:40.979770 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:40.979770 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 443/873] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:40.985693 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7296afab-fc2d-4d12-9beb-16e2375f5ccc None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:40.999836 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173808c0-ddc7-4732-bc86-327b6aef82f1 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:41.017219 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:41.125803 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7296afab-fc2d-4d12-9beb-16e2375f5ccc None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:41.164422 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173808c0-ddc7-4732-bc86-327b6aef82f1 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:41.194273 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6575b-188e-459a-a445-da5f48fc1c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:41.195292 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6575b-188e-459a-a445-da5f48fc1c1e None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:41.195966 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6575b-188e-459a-a445-da5f48fc1c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:41.196459 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6575b-188e-459a-a445-da5f48fc1c1e None admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:41.197897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce6575b-188e-459a-a445-da5f48fc1c1e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:41.198895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce6575b-188e-459a-a445-da5f48fc1c1e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:41.201599 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce6575b-188e-459a-a445-da5f48fc1c1e None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:41.223847 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f08e913-533f-4c81-b020-40e90126ffdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:41.224870 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08e913-533f-4c81-b020-40e90126ffdf None admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:41.225356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08e913-533f-4c81-b020-40e90126ffdf None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:41.225826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08e913-533f-4c81-b020-40e90126ffdf None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:41.226630 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 431/874] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:41.228869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f08e913-533f-4c81-b020-40e90126ffdf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:41.231309 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f08e913-533f-4c81-b020-40e90126ffdf None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:41.233722 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6575b-188e-459a-a445-da5f48fc1c1e None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:41.255320 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 444/875] 10.222.0.39 () {60 vars in 1141 bytes} [Fri Aug 5 19:50:40 2022] POST /identity/v3/services => generated 269 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:50:41.362305 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:41.384073 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6575b-188e-459a-a445-da5f48fc1c1e None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:41.434631 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce3e70-3507-4ba5-a1bd-214eff0d3791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:41.435592 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce3e70-3507-4ba5-a1bd-214eff0d3791 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:41.436080 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce3e70-3507-4ba5-a1bd-214eff0d3791 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:41.436355 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce3e70-3507-4ba5-a1bd-214eff0d3791 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:41.437192 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce3e70-3507-4ba5-a1bd-214eff0d3791 None admin] RBAC: Adding query filter params (role.id=7ebfd9e58fe549f08cdaec5f906a38a6, scope.project.id=80379e2104644e98a2e85d4e4af1b365, user.id=3488a7d2258242c0859a105c1d162767) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:41.437524 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce3e70-3507-4ba5-a1bd-214eff0d3791 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:41.438990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce3e70-3507-4ba5-a1bd-214eff0d3791 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:41.446253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 432/876] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:50:41 2022] GET /identity/v3/role_assignments?user.id=3488a7d2258242c0859a105c1d162767&scope.project.id=80379e2104644e98a2e85d4e4af1b365&role.id=7ebfd9e58fe549f08cdaec5f906a38a6 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:41.738585 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7676b-039f-4694-875d-bdc48fc30603 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:41.738864 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7676b-039f-4694-875d-bdc48fc30603 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:41.739138 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7676b-039f-4694-875d-bdc48fc30603 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:41.740856 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 445/877] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:41.747147 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c01294-7e75-416c-a90e-9f6f81e14820 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:41.747632 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c01294-7e75-416c-a90e-9f6f81e14820 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:41.747739 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c01294-7e75-416c-a90e-9f6f81e14820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:41.766787 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-f9c01294-7e75-416c-a90e-9f6f81e14820 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:41.767965 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-f9c01294-7e75-416c-a90e-9f6f81e14820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:41.852718 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 433/878] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:41 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:41.868097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7676b-039f-4694-875d-bdc48fc30603 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:41.896236 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:41.985591 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7676b-039f-4694-875d-bdc48fc30603 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:42.035161 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f04ee-da93-4a67-893c-0a79e7d23fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:42.035732 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f04ee-da93-4a67-893c-0a79e7d23fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:42.036015 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f04ee-da93-4a67-893c-0a79e7d23fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:42.036289 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f04ee-da93-4a67-893c-0a79e7d23fc2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:42.037076 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f04ee-da93-4a67-893c-0a79e7d23fc2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:42.038832 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f04ee-da93-4a67-893c-0a79e7d23fc2 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:42.049056 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-eb7f04ee-da93-4a67-893c-0a79e7d23fc2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:50:42.049056 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:42.049056 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:42.049056 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:42.049056 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:42.049056 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:42.049056 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:42.049056 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:42.049056 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:50:42.049056 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:42.049056 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:42.049056 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:42.049056 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:42.049056 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:42.049056 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:42.049056 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:42.049056 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:42.050109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:42.050109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:42.050109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:42.050109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:42.050109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:42.050109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:42.050109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:42.050109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:42.050109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:42.050109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:42.050109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:42.050109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:42.050109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:42.050109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:42.050109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:42.050109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:42.050946 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:42.050946 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:42.050946 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:42.050946 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:42.050946 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:42.050946 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:42.050946 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:42.050946 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:42.050946 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:42.050946 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:42.050946 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:42.050946 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:42.050946 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:50:42.050946 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:42.050946 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 446/879] 10.222.0.39 () {56 vars in 1126 bytes} [Fri Aug 5 19:50:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:42.057066 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c01294-7e75-416c-a90e-9f6f81e14820 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:42.143043 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:42.167960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c01294-7e75-416c-a90e-9f6f81e14820 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:42.204724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d687e-2579-4f6e-96d0-bae1638bc07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:42.205345 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d687e-2579-4f6e-96d0-bae1638bc07b None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:42.205617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d687e-2579-4f6e-96d0-bae1638bc07b None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:42.205890 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d687e-2579-4f6e-96d0-bae1638bc07b None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:42.206629 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d687e-2579-4f6e-96d0-bae1638bc07b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:42.206935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d687e-2579-4f6e-96d0-bae1638bc07b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:42.208301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d687e-2579-4f6e-96d0-bae1638bc07b None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:42.218902 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 434/880] 10.222.0.39 () {56 vars in 1122 bytes} [Fri Aug 5 19:50:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:42.220321 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f04ee-da93-4a67-893c-0a79e7d23fc2 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:42.291447 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f8001-5279-4c37-a842-78c930c80af1 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:42.291661 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f8001-5279-4c37-a842-78c930c80af1 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:42.291936 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f8001-5279-4c37-a842-78c930c80af1 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:42.293865 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 435/881] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:42.298819 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210cb79-0a3e-460d-a11c-b6e911cdd222 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:42.299100 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210cb79-0a3e-460d-a11c-b6e911cdd222 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:42.299374 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210cb79-0a3e-460d-a11c-b6e911cdd222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:42.319969 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-b210cb79-0a3e-460d-a11c-b6e911cdd222 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:42.321543 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-b210cb79-0a3e-460d-a11c-b6e911cdd222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:42.327167 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:42.347461 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f04ee-da93-4a67-893c-0a79e7d23fc2 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:42.383208 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695aac26-2cb1-4a00-9f23-9fca37f67548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:42.384010 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695aac26-2cb1-4a00-9f23-9fca37f67548 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:42.384319 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695aac26-2cb1-4a00-9f23-9fca37f67548 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:42.384611 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695aac26-2cb1-4a00-9f23-9fca37f67548 None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:42.385846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695aac26-2cb1-4a00-9f23-9fca37f67548 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:42.386205 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695aac26-2cb1-4a00-9f23-9fca37f67548 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:42.388828 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695aac26-2cb1-4a00-9f23-9fca37f67548 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:42.394726 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 447/882] 10.222.0.39 () {56 vars in 1122 bytes} [Fri Aug 5 19:50:42 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:42.417520 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 436/883] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:42 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:50:42.440809 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695aac26-2cb1-4a00-9f23-9fca37f67548 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:42.576623 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695aac26-2cb1-4a00-9f23-9fca37f67548 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:42.636914 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226e5836-318d-4922-9718-d01c5c2ca7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:42.637761 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e5836-318d-4922-9718-d01c5c2ca7ce None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:42.638042 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e5836-318d-4922-9718-d01c5c2ca7ce None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:42.638315 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e5836-318d-4922-9718-d01c5c2ca7ce None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:42.639460 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226e5836-318d-4922-9718-d01c5c2ca7ce None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:42.640981 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226e5836-318d-4922-9718-d01c5c2ca7ce None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:42.652416 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-226e5836-318d-4922-9718-d01c5c2ca7ce None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 19:50:42.652416 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:42.652416 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:42.652416 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:42.652416 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:42.652416 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:42.652416 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:42.652416 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:42.652416 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 19:50:42.652416 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:42.652416 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:42.652416 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:42.652416 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:42.652416 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:42.652416 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:42.652416 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:42.652416 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:42.653501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:42.653501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:42.653501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:42.653501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:42.653501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:42.653501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:42.653501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:42.653501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:42.653501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:42.653501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:42.653501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:42.653501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:42.653501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:42.653501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:42.653501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:42.653501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:42.654334 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:42.654334 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:42.654334 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:42.654334 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:42.654334 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:42.654334 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:42.654334 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:42.654334 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:42.654334 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:42.654334 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:42.654334 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:42.654334 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:42.654334 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 19:50:42.654334 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:42.654334 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 448/884] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Aug 5 19:50:42 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:42.661793 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b210cb79-0a3e-460d-a11c-b6e911cdd222 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:42.739166 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:42.774494 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b210cb79-0a3e-460d-a11c-b6e911cdd222 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:42.814453 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a11481-e3ce-4148-ad50-dbab4773af1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:42.815558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a11481-e3ce-4148-ad50-dbab4773af1e None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:42.816056 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a11481-e3ce-4148-ad50-dbab4773af1e None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:42.816539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a11481-e3ce-4148-ad50-dbab4773af1e None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:42.817486 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a11481-e3ce-4148-ad50-dbab4773af1e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:42.818085 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a11481-e3ce-4148-ad50-dbab4773af1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:42.819727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a11481-e3ce-4148-ad50-dbab4773af1e None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:42.827297 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 437/885] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Aug 5 19:50:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:42.833639 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226e5836-318d-4922-9718-d01c5c2ca7ce None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:42.858849 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:42.972691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226e5836-318d-4922-9718-d01c5c2ca7ce None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:43.009550 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0cb3a4-6afb-4f7d-a193-808280b56f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:43.009550 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0cb3a4-6afb-4f7d-a193-808280b56f12 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:43.010310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0cb3a4-6afb-4f7d-a193-808280b56f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:43.010310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0cb3a4-6afb-4f7d-a193-808280b56f12 None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:43.011308 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0cb3a4-6afb-4f7d-a193-808280b56f12 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:43.011628 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0cb3a4-6afb-4f7d-a193-808280b56f12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:43.013156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0cb3a4-6afb-4f7d-a193-808280b56f12 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:43.019011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 449/886] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Aug 5 19:50:42 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:43.025005 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a11481-e3ce-4148-ad50-dbab4773af1e None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:43.141870 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a11481-e3ce-4148-ad50-dbab4773af1e None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:43.181652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e8ba5-5fa3-4dd9-84ce-967b4aa2143e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:43.182428 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e8ba5-5fa3-4dd9-84ce-967b4aa2143e None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:43.182627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e8ba5-5fa3-4dd9-84ce-967b4aa2143e None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:43.183068 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e8ba5-5fa3-4dd9-84ce-967b4aa2143e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:43.183879 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e8ba5-5fa3-4dd9-84ce-967b4aa2143e None admin] RBAC: Adding query filter params (interface=public, service_id=f11faa9a3e534da092be93fea722ed7e, region_id=RegionOne) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:43.184190 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e8ba5-5fa3-4dd9-84ce-967b4aa2143e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:43.185594 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e8ba5-5fa3-4dd9-84ce-967b4aa2143e None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:43.188387 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:43.192324 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 438/887] 10.222.0.39 () {56 vars in 1251 bytes} [Fri Aug 5 19:50:43 2022] GET /identity/v3/endpoints?service_id=f11faa9a3e534da092be93fea722ed7e&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:43.198301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0cb3a4-6afb-4f7d-a193-808280b56f12 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:43.284461 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:43.315479 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0cb3a4-6afb-4f7d-a193-808280b56f12 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:43.358467 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3fa336-be7a-4ffe-b223-871f094ec1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:43.359180 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3fa336-be7a-4ffe-b223-871f094ec1bf None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:43.359526 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3fa336-be7a-4ffe-b223-871f094ec1bf None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:43.359913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3fa336-be7a-4ffe-b223-871f094ec1bf None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:43.360775 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3fa336-be7a-4ffe-b223-871f094ec1bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:43.363178 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3fa336-be7a-4ffe-b223-871f094ec1bf None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:43.371088 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 450/888] 10.222.0.39 () {56 vars in 1087 bytes} [Fri Aug 5 19:50:43 2022] GET /identity/v3/services => generated 1268 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:50:43.505909 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71201386-e43b-4123-aec4-46a44468c453 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:43.506107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71201386-e43b-4123-aec4-46a44468c453 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:43.506368 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71201386-e43b-4123-aec4-46a44468c453 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:43.508105 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 439/889] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:43.514238 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc88503-1b9f-4cb7-b75d-1e4705738f04 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:43.514573 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc88503-1b9f-4cb7-b75d-1e4705738f04 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:43.514787 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc88503-1b9f-4cb7-b75d-1e4705738f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:43.538726 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-cfc88503-1b9f-4cb7-b75d-1e4705738f04 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:43.538726 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-cfc88503-1b9f-4cb7-b75d-1e4705738f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:43.627537 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 451/890] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:43 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:43.653734 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71201386-e43b-4123-aec4-46a44468c453 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:43.782308 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71201386-e43b-4123-aec4-46a44468c453 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:43.809760 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1ae47-bda9-4d86-a5c5-33fe33deeffd None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:43.810392 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1ae47-bda9-4d86-a5c5-33fe33deeffd None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:43.810874 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1ae47-bda9-4d86-a5c5-33fe33deeffd None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:43.812705 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 452/891] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:43.818181 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095dec3f-3e35-469f-8f83-d00f89853c73 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:43.818742 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095dec3f-3e35-469f-8f83-d00f89853c73 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:43.819213 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf52c93-323f-4d11-9d78-4d652394bef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:43.819468 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095dec3f-3e35-469f-8f83-d00f89853c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:43.819852 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf52c93-323f-4d11-9d78-4d652394bef2 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:43.820130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf52c93-323f-4d11-9d78-4d652394bef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:43.820397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf52c93-323f-4d11-9d78-4d652394bef2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:43.821043 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf52c93-323f-4d11-9d78-4d652394bef2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:43.822401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf52c93-323f-4d11-9d78-4d652394bef2 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:43.833059 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-4cf52c93-323f-4d11-9d78-4d652394bef2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 19:50:43.833059 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:43.833059 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:43.833059 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:43.833059 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:43.833059 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:43.833059 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:43.833059 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:43.833059 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 19:50:43.833059 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:43.833059 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:43.833059 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:43.833059 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:43.833059 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:43.833059 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:43.833059 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:43.834046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:43.834046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:43.834046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:43.834046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:43.834046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:43.834046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:43.834046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:43.834046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:43.834046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:43.834046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:43.834046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:43.834046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:43.834046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:43.834046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:43.834046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:43.834046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:43.834874 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:43.834874 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:43.834874 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:43.834874 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:43.834874 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:43.834874 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:43.834874 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:43.834874 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:43.834874 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:43.834874 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:43.834874 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:43.834874 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:43.834874 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:43.834874 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 19:50:43.834874 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:43.834874 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 440/892] 10.222.0.39 () {56 vars in 1132 bytes} [Fri Aug 5 19:50:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:43.841288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf52c93-323f-4d11-9d78-4d652394bef2 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:43.844833 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-095dec3f-3e35-469f-8f83-d00f89853c73 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:43.846290 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-095dec3f-3e35-469f-8f83-d00f89853c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:43.924420 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 453/893] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:43 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:43.924816 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:43.948687 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095dec3f-3e35-469f-8f83-d00f89853c73 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:43.951297 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:43.964560 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf52c93-323f-4d11-9d78-4d652394bef2 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:44.003134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6643665c-d5ac-49d8-95dd-1b9814bdeb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:44.003912 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643665c-d5ac-49d8-95dd-1b9814bdeb9a None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:44.004313 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643665c-d5ac-49d8-95dd-1b9814bdeb9a None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:44.004728 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643665c-d5ac-49d8-95dd-1b9814bdeb9a None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:44.005522 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6643665c-d5ac-49d8-95dd-1b9814bdeb9a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:44.005927 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6643665c-d5ac-49d8-95dd-1b9814bdeb9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:44.007557 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6643665c-d5ac-49d8-95dd-1b9814bdeb9a None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:44.018806 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 441/894] 10.222.0.39 () {56 vars in 1126 bytes} [Fri Aug 5 19:50:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:44.020139 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6643665c-d5ac-49d8-95dd-1b9814bdeb9a None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:44.101755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095dec3f-3e35-469f-8f83-d00f89853c73 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:44.174254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6643665c-d5ac-49d8-95dd-1b9814bdeb9a None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:44.175380 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953c23b-e437-437c-98fe-677b113b6043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:44.178187 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953c23b-e437-437c-98fe-677b113b6043 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:44.178982 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953c23b-e437-437c-98fe-677b113b6043 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:44.179776 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953c23b-e437-437c-98fe-677b113b6043 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:44.205248 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:44.207002 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4953c23b-e437-437c-98fe-677b113b6043 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:44.216752 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4134a-83cb-4851-b202-f6015bde4ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:44.217371 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4134a-83cb-4851-b202-f6015bde4ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:44.217661 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4134a-83cb-4851-b202-f6015bde4ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:44.217919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4134a-83cb-4851-b202-f6015bde4ee8 None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:44.218593 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc4134a-83cb-4851-b202-f6015bde4ee8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:44.218898 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc4134a-83cb-4851-b202-f6015bde4ee8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:44.220296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc4134a-83cb-4851-b202-f6015bde4ee8 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:44.222426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-4953c23b-e437-437c-98fe-677b113b6043 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:50:44.222426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:44.222426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:44.222426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:44.222426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:44.222426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:44.222426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:44.222426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:44.222426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:44.222426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:44.222426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:44.222426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:44.222426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:44.222426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:44.222426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:44.222426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:44.224171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:44.224171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:44.224171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:44.224171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:44.224171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:44.224171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:44.224171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:44.224171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:44.224171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:44.224171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:44.224171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:44.224171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:44.224171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:44.224171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:44.224171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:44.224171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:44.226839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:44.226839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:44.226839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:44.226839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:44.226839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:44.226839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:44.226839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:44.226839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:44.226839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:44.226839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:44.226839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:50:44.226839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:44.232363 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 454/895] 10.222.0.39 () {56 vars in 1120 bytes} [Fri Aug 5 19:50:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:44.236611 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 442/896] 10.222.0.39 () {56 vars in 1126 bytes} [Fri Aug 5 19:50:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:44.238309 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953c23b-e437-437c-98fe-677b113b6043 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:44.240249 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4134a-83cb-4851-b202-f6015bde4ee8 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:44.322738 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:44.368857 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953c23b-e437-437c-98fe-677b113b6043 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:44.383557 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4134a-83cb-4851-b202-f6015bde4ee8 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:44.408719 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c0a1be-1c67-45d1-a552-6a5ee6695036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:44.409605 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0a1be-1c67-45d1-a552-6a5ee6695036 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:44.410050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0a1be-1c67-45d1-a552-6a5ee6695036 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:44.410469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0a1be-1c67-45d1-a552-6a5ee6695036 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:44.411468 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c0a1be-1c67-45d1-a552-6a5ee6695036 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:44.411925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c0a1be-1c67-45d1-a552-6a5ee6695036 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:44.413484 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c0a1be-1c67-45d1-a552-6a5ee6695036 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:44.428217 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 455/897] 10.222.0.39 () {56 vars in 1115 bytes} [Fri Aug 5 19:50:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:44.429624 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c0a1be-1c67-45d1-a552-6a5ee6695036 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:44.431282 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ea108-2892-4da5-9772-6927f620dff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:44.432546 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ea108-2892-4da5-9772-6927f620dff5 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:44.433064 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ea108-2892-4da5-9772-6927f620dff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:44.433732 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ea108-2892-4da5-9772-6927f620dff5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:44.434995 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0ea108-2892-4da5-9772-6927f620dff5 None admin] RBAC: Adding query filter params (interface=public, service_id=05a620035d3b4c7299dcbe3c20b3af6a, region_id=RegionOne) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:44.436463 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0ea108-2892-4da5-9772-6927f620dff5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:44.438065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0ea108-2892-4da5-9772-6927f620dff5 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:44.448736 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 443/898] 10.222.0.39 () {56 vars in 1251 bytes} [Fri Aug 5 19:50:44 2022] GET /identity/v3/endpoints?service_id=05a620035d3b4c7299dcbe3c20b3af6a&interface=public®ion_id=RegionOne => generated 194 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:44.452257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ea108-2892-4da5-9772-6927f620dff5 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:44.538696 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:44.583037 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c0a1be-1c67-45d1-a552-6a5ee6695036 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:44.603360 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ea108-2892-4da5-9772-6927f620dff5 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:44.616562 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:44.633139 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bbb88d-ddc6-424c-b684-ddd60bba8fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:44.634146 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bbb88d-ddc6-424c-b684-ddd60bba8fea None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:44.634440 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bbb88d-ddc6-424c-b684-ddd60bba8fea None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:44.635014 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bbb88d-ddc6-424c-b684-ddd60bba8fea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:44.642751 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b8d793-23e0-4cf8-9004-242faf1d50bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:44.643374 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8d793-23e0-4cf8-9004-242faf1d50bd None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:44.643654 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8d793-23e0-4cf8-9004-242faf1d50bd None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:44.643927 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8d793-23e0-4cf8-9004-242faf1d50bd None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:44.644585 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b8d793-23e0-4cf8-9004-242faf1d50bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:44.646035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b8d793-23e0-4cf8-9004-242faf1d50bd None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:44.649660 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bbb88d-ddc6-424c-b684-ddd60bba8fea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:44.651819 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bbb88d-ddc6-424c-b684-ddd60bba8fea None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:44.654127 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 444/899] 10.222.0.39 () {56 vars in 1087 bytes} [Fri Aug 5 19:50:44 2022] GET /identity/v3/services => generated 1268 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:50:44.666218 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31660889-c771-4820-927b-4d7a761e0326 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:44.666567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31660889-c771-4820-927b-4d7a761e0326 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:44.666832 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31660889-c771-4820-927b-4d7a761e0326 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:44.668710 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 445/900] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:44.674576 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e8acf-7d55-4dc5-b58a-5c8deb18b0c3 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:44.674867 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e8acf-7d55-4dc5-b58a-5c8deb18b0c3 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:44.675138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e8acf-7d55-4dc5-b58a-5c8deb18b0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:44.679445 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-53bbb88d-ddc6-424c-b684-ddd60bba8fea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:44.679445 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:44.679445 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:44.679445 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:44.679445 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:44.679445 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:44.679445 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:44.679445 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:44.679445 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:44.679445 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:44.679445 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:44.679445 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:44.679445 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:44.679445 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:44.679445 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:44.680436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:44.680436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:44.680436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:44.680436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:44.680436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:44.680436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:44.680436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:44.680436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:44.680436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:44.680436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:44.680436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:44.680436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:44.680436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:44.680436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:44.680436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:44.680436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:44.680436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:44.681382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:44.681382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:44.681382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:44.681382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:44.683933 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 456/901] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:44.689763 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bbb88d-ddc6-424c-b684-ddd60bba8fea None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:44.696293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-103e8acf-7d55-4dc5-b58a-5c8deb18b0c3 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:44.698648 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-103e8acf-7d55-4dc5-b58a-5c8deb18b0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:44.798062 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 446/902] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:44 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:50:44.824135 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e8acf-7d55-4dc5-b58a-5c8deb18b0c3 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:44.833841 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bbb88d-ddc6-424c-b684-ddd60bba8fea None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:44.886780 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f4a9b-f5f3-4273-b54a-8c99fcf04580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:44.888218 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f4a9b-f5f3-4273-b54a-8c99fcf04580 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:44.888964 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f4a9b-f5f3-4273-b54a-8c99fcf04580 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:44.889423 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f4a9b-f5f3-4273-b54a-8c99fcf04580 None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:44.890616 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f4a9b-f5f3-4273-b54a-8c99fcf04580 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:44.891615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:44.892001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f4a9b-f5f3-4273-b54a-8c99fcf04580 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:44.893584 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f4a9b-f5f3-4273-b54a-8c99fcf04580 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:44.913227 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 457/903] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:44.920625 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f4a9b-f5f3-4273-b54a-8c99fcf04580 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:44.948442 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e8acf-7d55-4dc5-b58a-5c8deb18b0c3 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:44.957496 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:44.987774 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d54e6-d64e-44e5-a9a5-ca8229a63314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:44.988730 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d54e6-d64e-44e5-a9a5-ca8229a63314 None admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:44.989240 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d54e6-d64e-44e5-a9a5-ca8229a63314 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:44.989773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d54e6-d64e-44e5-a9a5-ca8229a63314 None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:44.990714 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d54e6-d64e-44e5-a9a5-ca8229a63314 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:44.992279 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d54e6-d64e-44e5-a9a5-ca8229a63314 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:45.011597 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 447/904] 10.222.0.39 () {60 vars in 1141 bytes} [Fri Aug 5 19:50:44 2022] POST /identity/v3/services => generated 241 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:50:45.045923 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f4a9b-f5f3-4273-b54a-8c99fcf04580 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:45.095348 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf1850-1d7a-4d28-b148-3f5163bec5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:45.096060 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf1850-1d7a-4d28-b148-3f5163bec5b7 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:45.096406 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf1850-1d7a-4d28-b148-3f5163bec5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:45.096719 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf1850-1d7a-4d28-b148-3f5163bec5b7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:45.105666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf1850-1d7a-4d28-b148-3f5163bec5b7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:45.107156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf1850-1d7a-4d28-b148-3f5163bec5b7 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:45.121507 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-eccf1850-1d7a-4d28-b148-3f5163bec5b7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:50:45.121507 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:45.121507 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:45.121507 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:45.121507 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:45.121507 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:45.121507 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:45.121507 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:45.121507 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:45.121507 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:45.121507 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:45.121507 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:45.121507 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:45.121507 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:45.121507 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:45.121507 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:45.122937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:45.122937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:45.122937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:45.122937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:45.122937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:45.122937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:45.122937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:45.122937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:45.122937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:45.122937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:45.122937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:45.122937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:45.122937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:45.122937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:45.122937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:45.122937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:45.122937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:45.122937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:45.125075 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:45.125075 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:45.125075 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:45.125075 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:45.125075 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:45.125075 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:45.125075 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:45.125075 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:45.125075 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:50:45.125075 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:45.125075 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 458/905] 10.222.0.39 () {56 vars in 1117 bytes} [Fri Aug 5 19:50:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:45.129415 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d54e6-d64e-44e5-a9a5-ca8229a63314 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:45.248436 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d54e6-d64e-44e5-a9a5-ca8229a63314 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:45.259625 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:45.288627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011bb36f-370b-45f0-9348-a0cb4b567fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:45.289255 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bb36f-370b-45f0-9348-a0cb4b567fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:45.289577 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bb36f-370b-45f0-9348-a0cb4b567fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:45.289848 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bb36f-370b-45f0-9348-a0cb4b567fa9 None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:45.290648 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011bb36f-370b-45f0-9348-a0cb4b567fa9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:45.290951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011bb36f-370b-45f0-9348-a0cb4b567fa9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:45.293214 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011bb36f-370b-45f0-9348-a0cb4b567fa9 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:45.313507 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 448/906] 10.222.0.39 () {56 vars in 1149 bytes} [Fri Aug 5 19:50:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:45.318714 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf1850-1d7a-4d28-b148-3f5163bec5b7 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:45.421026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:45.426917 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf1850-1d7a-4d28-b148-3f5163bec5b7 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:45.464897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fabeaa5-c228-450e-b144-805205542fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:45.465519 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabeaa5-c228-450e-b144-805205542fbd None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:45.465798 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabeaa5-c228-450e-b144-805205542fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:45.466070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabeaa5-c228-450e-b144-805205542fbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:45.539893 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fabeaa5-c228-450e-b144-805205542fbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:45.542060 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fabeaa5-c228-450e-b144-805205542fbd None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:45.641297 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:45.649856 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 459/907] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:50:45 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:50:45.657817 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011bb36f-370b-45f0-9348-a0cb4b567fa9 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:45.761938 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f8abd-a76d-484e-ad0b-9367daf5d191 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:45.762136 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f8abd-a76d-484e-ad0b-9367daf5d191 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:45.762399 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f8abd-a76d-484e-ad0b-9367daf5d191 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:45.764614 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 460/908] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:45.770244 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a11dc-6eb5-438d-8ddf-2a6ae7af82df None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:45.770820 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a11dc-6eb5-438d-8ddf-2a6ae7af82df None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:45.771755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a11dc-6eb5-438d-8ddf-2a6ae7af82df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:45.777975 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011bb36f-370b-45f0-9348-a0cb4b567fa9 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:45.798757 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-788a11dc-6eb5-438d-8ddf-2a6ae7af82df None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:45.800518 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-788a11dc-6eb5-438d-8ddf-2a6ae7af82df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:45.817079 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7473d36-4e24-465d-b67c-f7d0d410939b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:45.817800 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7473d36-4e24-465d-b67c-f7d0d410939b None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:45.818107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7473d36-4e24-465d-b67c-f7d0d410939b None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:45.818250 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7473d36-4e24-465d-b67c-f7d0d410939b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:45.832624 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7473d36-4e24-465d-b67c-f7d0d410939b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:45.832624 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7473d36-4e24-465d-b67c-f7d0d410939b None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:45.851634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-c7473d36-4e24-465d-b67c-f7d0d410939b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:45.851634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:45.851634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:45.851634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:45.851634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:45.851634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:45.851634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:45.851634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:45.851634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:45.851634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:45.851634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:45.851634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:45.851634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:45.851634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:45.851634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:45.852604 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:45.852604 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:45.852604 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:45.852604 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:45.852604 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:45.852604 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:45.852604 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:45.852604 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:45.852604 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:45.852604 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:45.852604 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:45.852604 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:45.852604 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:45.852604 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:45.852604 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:45.852604 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:45.852604 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:45.853559 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:45.853559 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:45.853559 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:45.853559 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:45.853559 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 449/909] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:45.862010 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7473d36-4e24-465d-b67c-f7d0d410939b None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:45.892798 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:45.902623 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 461/910] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:50:45.929231 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a11dc-6eb5-438d-8ddf-2a6ae7af82df None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:46.013994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7473d36-4e24-465d-b67c-f7d0d410939b None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:46.063324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a11dc-6eb5-438d-8ddf-2a6ae7af82df None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:46.065917 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b2310-639a-436e-8551-98478b65e980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:46.066847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b2310-639a-436e-8551-98478b65e980 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:46.067198 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b2310-639a-436e-8551-98478b65e980 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:46.067703 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b2310-639a-436e-8551-98478b65e980 None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:46.068826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b2310-639a-436e-8551-98478b65e980 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:46.069374 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b2310-639a-436e-8551-98478b65e980 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:46.071402 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b2310-639a-436e-8551-98478b65e980 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:46.104022 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 450/911] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:46.106176 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:46.108242 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b2310-639a-436e-8551-98478b65e980 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:46.124603 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e8aa2-69c2-4d63-9812-169c402e9366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:46.124603 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e8aa2-69c2-4d63-9812-169c402e9366 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:46.124603 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e8aa2-69c2-4d63-9812-169c402e9366 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:46.124603 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e8aa2-69c2-4d63-9812-169c402e9366 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:46.125053 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e8aa2-69c2-4d63-9812-169c402e9366 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:46.125053 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e8aa2-69c2-4d63-9812-169c402e9366 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:46.133280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-5c9e8aa2-69c2-4d63-9812-169c402e9366 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 19:50:46.133280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:46.133280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:46.133280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:46.133280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:46.133280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:46.133280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:46.133280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:46.133280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 19:50:46.133280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:46.133280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:46.133280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:46.133280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:46.133280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:46.133280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:46.133280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:46.133280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:46.134407 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:46.134407 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:46.134407 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:46.134407 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:46.134407 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:46.134407 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:46.134407 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:46.134407 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:46.134407 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:46.134407 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:46.134407 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:46.134407 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:46.134407 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:46.134407 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:46.134407 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:46.134407 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:46.135868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:46.135868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:46.135868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:46.135868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:46.135868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:46.135868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:46.135868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:46.135868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:46.135868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:46.135868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:46.135868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:46.135868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:46.135868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 19:50:46.135868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:46.135868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 462/912] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Aug 5 19:50:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:46.143981 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e8aa2-69c2-4d63-9812-169c402e9366 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:46.212738 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:46.238830 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b2310-639a-436e-8551-98478b65e980 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:46.278784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e8aa2-69c2-4d63-9812-169c402e9366 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:46.281759 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdbc740-c905-4c9e-8af4-79813cdbd888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:46.282714 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbc740-c905-4c9e-8af4-79813cdbd888 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:46.283218 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbc740-c905-4c9e-8af4-79813cdbd888 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:46.283700 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbc740-c905-4c9e-8af4-79813cdbd888 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:46.295678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdbc740-c905-4c9e-8af4-79813cdbd888 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:46.295678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdbc740-c905-4c9e-8af4-79813cdbd888 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:46.311567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-7fdbc740-c905-4c9e-8af4-79813cdbd888 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:46.311567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:46.311567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:46.311567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:46.311567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:46.311567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:46.311567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:46.311567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:46.311567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:46.311567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:46.311567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:46.311567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:46.311567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:46.311567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:46.311567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:46.312464 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:46.312464 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:46.312464 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:46.312464 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:46.312464 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:46.312464 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:46.312464 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:46.312464 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:46.312464 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:46.312464 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:46.312464 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:46.312464 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:46.312464 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:46.312464 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:46.312464 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:46.312464 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:46.312464 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:46.313417 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:46.313417 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:46.313417 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:46.313417 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:46.313417 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:46.313417 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:46.313417 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:46.313417 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:46.315794 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 451/913] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:46.322316 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5705a-4bda-46c3-bbb4-9822f2a68419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:46.323585 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5705a-4bda-46c3-bbb4-9822f2a68419 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:46.323863 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5705a-4bda-46c3-bbb4-9822f2a68419 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:46.324119 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdbc740-c905-4c9e-8af4-79813cdbd888 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:46.324345 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5705a-4bda-46c3-bbb4-9822f2a68419 None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:46.324945 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a5705a-4bda-46c3-bbb4-9822f2a68419 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:46.325226 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a5705a-4bda-46c3-bbb4-9822f2a68419 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:46.326676 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a5705a-4bda-46c3-bbb4-9822f2a68419 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:46.332854 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 463/914] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Aug 5 19:50:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:46.341665 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5705a-4bda-46c3-bbb4-9822f2a68419 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:46.429142 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:46.469043 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdbc740-c905-4c9e-8af4-79813cdbd888 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:46.499740 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5705a-4bda-46c3-bbb4-9822f2a68419 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:46.519266 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398673b-fd5d-43d9-be77-fc4cb5cb5883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:46.520167 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398673b-fd5d-43d9-be77-fc4cb5cb5883 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:46.520621 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398673b-fd5d-43d9-be77-fc4cb5cb5883 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:46.521043 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398673b-fd5d-43d9-be77-fc4cb5cb5883 None admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:46.522167 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7398673b-fd5d-43d9-be77-fc4cb5cb5883 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:46.522643 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7398673b-fd5d-43d9-be77-fc4cb5cb5883 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:46.524634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7398673b-fd5d-43d9-be77-fc4cb5cb5883 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:46.543267 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb7a56-3176-4df8-bf4f-cd5c10cd5756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:46.548021 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb7a56-3176-4df8-bf4f-cd5c10cd5756 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:46.548021 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb7a56-3176-4df8-bf4f-cd5c10cd5756 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:46.548021 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb7a56-3176-4df8-bf4f-cd5c10cd5756 None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:46.548399 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:46.548494 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeb7a56-3176-4df8-bf4f-cd5c10cd5756 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:46.548494 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeb7a56-3176-4df8-bf4f-cd5c10cd5756 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:46.548494 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeb7a56-3176-4df8-bf4f-cd5c10cd5756 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:46.549250 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 452/915] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:46.551231 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398673b-fd5d-43d9-be77-fc4cb5cb5883 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:46.559403 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 464/916] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Aug 5 19:50:46 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:46.560803 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb7a56-3176-4df8-bf4f-cd5c10cd5756 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:46.673415 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398673b-fd5d-43d9-be77-fc4cb5cb5883 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:46.676729 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb7a56-3176-4df8-bf4f-cd5c10cd5756 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:46.693441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:46.716455 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8e6db6-8fd0-4f52-878a-8c23e7799dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:46.717366 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e6db6-8fd0-4f52-878a-8c23e7799dba None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:46.717815 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e6db6-8fd0-4f52-878a-8c23e7799dba None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:46.718230 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e6db6-8fd0-4f52-878a-8c23e7799dba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:46.719169 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8e6db6-8fd0-4f52-878a-8c23e7799dba None admin] RBAC: Adding query filter params (role.id=0a40666ef2c140d1951dea50f3127507, scope.project.id=80379e2104644e98a2e85d4e4af1b365, user.id=3488a7d2258242c0859a105c1d162767) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:46.719306 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd6d4cc-610a-4bcd-9929-68e9c7280726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:46.719759 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8e6db6-8fd0-4f52-878a-8c23e7799dba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:46.720090 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6d4cc-610a-4bcd-9929-68e9c7280726 None admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:46.720470 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6d4cc-610a-4bcd-9929-68e9c7280726 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:46.720872 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6d4cc-610a-4bcd-9929-68e9c7280726 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:46.721690 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd6d4cc-610a-4bcd-9929-68e9c7280726 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:46.722041 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8e6db6-8fd0-4f52-878a-8c23e7799dba None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:46.723251 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd6d4cc-610a-4bcd-9929-68e9c7280726 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:46.736709 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 453/917] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:50:46 2022] GET /identity/v3/role_assignments?user.id=3488a7d2258242c0859a105c1d162767&scope.project.id=80379e2104644e98a2e85d4e4af1b365&role.id=0a40666ef2c140d1951dea50f3127507 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:46.764159 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 465/918] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:46 2022] POST /identity/v3/endpoints => generated 346 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:46.925657 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d9ac0-c348-4525-8862-fc9d2ea27985 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:46.926192 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d9ac0-c348-4525-8862-fc9d2ea27985 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:46.926192 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d9ac0-c348-4525-8862-fc9d2ea27985 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:46.927916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 454/919] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:46.934596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a1cc58-e6c0-4333-8f5e-1ba2c563f6eb None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:46.935316 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a1cc58-e6c0-4333-8f5e-1ba2c563f6eb None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:46.935813 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a1cc58-e6c0-4333-8f5e-1ba2c563f6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:46.958737 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-08a1cc58-e6c0-4333-8f5e-1ba2c563f6eb None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:46.960398 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-08a1cc58-e6c0-4333-8f5e-1ba2c563f6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:47.075588 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 466/920] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:47.101488 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d9ac0-c348-4525-8862-fc9d2ea27985 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:47.200866 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:47.260015 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d9ac0-c348-4525-8862-fc9d2ea27985 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:47.295986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c651a-4039-4bac-aa25-82c8b68004f9 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:47.296570 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c651a-4039-4bac-aa25-82c8b68004f9 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:47.296728 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c651a-4039-4bac-aa25-82c8b68004f9 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:47.299777 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 467/921] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:50:47.304626 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa36eb1f-b4e8-4205-8d4e-407a29846913 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:47.304937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa36eb1f-b4e8-4205-8d4e-407a29846913 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:47.305220 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa36eb1f-b4e8-4205-8d4e-407a29846913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:47.317304 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91156b1f-c9fe-4c78-86c4-613473eff27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:47.318442 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91156b1f-c9fe-4c78-86c4-613473eff27d None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:47.319083 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91156b1f-c9fe-4c78-86c4-613473eff27d None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:47.319576 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91156b1f-c9fe-4c78-86c4-613473eff27d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:47.320661 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91156b1f-c9fe-4c78-86c4-613473eff27d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:47.322543 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91156b1f-c9fe-4c78-86c4-613473eff27d None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:47.326867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-aa36eb1f-b4e8-4205-8d4e-407a29846913 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:47.328345 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-aa36eb1f-b4e8-4205-8d4e-407a29846913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:47.335711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-91156b1f-c9fe-4c78-86c4-613473eff27d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 19:50:47.335711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:47.335711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:47.335711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:47.335711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:47.335711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:47.335711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:47.335711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:47.335711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 19:50:47.335711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:47.335711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:47.335711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:47.335711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:47.335711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:47.335711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:47.335711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:47.337072 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:47.337072 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:47.337072 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:47.337072 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:47.337072 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:47.337072 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:47.337072 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:47.337072 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:47.337072 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:47.337072 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:47.337072 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:47.337072 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:47.337072 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:47.337072 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:47.337072 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:47.337072 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:47.337935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:47.337935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:47.337935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:47.337935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:47.337935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:47.337935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:47.337935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:47.337935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:47.337935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:47.337935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:47.337935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:47.337935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:47.337935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:47.337935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 19:50:47.337935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:47.337935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 455/922] 10.222.0.39 () {56 vars in 1132 bytes} [Fri Aug 5 19:50:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:47.343999 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91156b1f-c9fe-4c78-86c4-613473eff27d None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:47.430039 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 468/923] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:50:47.462569 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa36eb1f-b4e8-4205-8d4e-407a29846913 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:47.488807 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91156b1f-c9fe-4c78-86c4-613473eff27d None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:47.493706 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:47.530115 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:47.542121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5ead3-0980-44e8-9468-3801afd7bd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:47.544062 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5ead3-0980-44e8-9468-3801afd7bd10 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:47.544710 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5ead3-0980-44e8-9468-3801afd7bd10 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:47.545208 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5ead3-0980-44e8-9468-3801afd7bd10 None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:47.546114 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5ead3-0980-44e8-9468-3801afd7bd10 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:47.546628 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5ead3-0980-44e8-9468-3801afd7bd10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:47.548781 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5ead3-0980-44e8-9468-3801afd7bd10 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:47.561941 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 456/924] 10.222.0.39 () {56 vars in 1126 bytes} [Fri Aug 5 19:50:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:47.563129 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5ead3-0980-44e8-9468-3801afd7bd10 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:47.587974 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa36eb1f-b4e8-4205-8d4e-407a29846913 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:47.637364 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5db44a-4adc-4531-8095-a78954a9b868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:47.638487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5db44a-4adc-4531-8095-a78954a9b868 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:47.639073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5db44a-4adc-4531-8095-a78954a9b868 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:47.639560 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5db44a-4adc-4531-8095-a78954a9b868 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:47.640477 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5db44a-4adc-4531-8095-a78954a9b868 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:47.642195 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5db44a-4adc-4531-8095-a78954a9b868 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:47.654602 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-1c5db44a-4adc-4531-8095-a78954a9b868 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:50:47.654602 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:47.654602 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:47.654602 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:47.654602 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:47.654602 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:47.654602 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:47.654602 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:47.654602 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:50:47.654602 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:47.654602 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:47.654602 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:47.654602 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:47.654602 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:47.654602 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:47.654602 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:47.654602 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:47.655696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:47.655696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:47.655696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:47.655696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:47.655696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:47.655696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:47.655696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:47.655696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:47.655696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:47.655696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:47.655696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:47.655696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:47.655696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:47.655696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:47.655696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:47.655696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:47.656546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:47.656546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:47.656546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:47.656546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:47.656546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:47.656546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:47.656546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:47.656546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:47.656546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:47.656546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:47.656546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:47.656546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:47.656546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:50:47.656546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:47.664617 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 469/925] 10.222.0.39 () {56 vars in 1126 bytes} [Fri Aug 5 19:50:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:47.665927 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5db44a-4adc-4531-8095-a78954a9b868 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:47.682797 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5ead3-0980-44e8-9468-3801afd7bd10 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:47.720204 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94992ce4-b602-4c57-a299-36eb9496bccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:47.720874 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94992ce4-b602-4c57-a299-36eb9496bccb None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:47.721177 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94992ce4-b602-4c57-a299-36eb9496bccb None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:47.721449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94992ce4-b602-4c57-a299-36eb9496bccb None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:47.722188 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94992ce4-b602-4c57-a299-36eb9496bccb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:47.722535 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94992ce4-b602-4c57-a299-36eb9496bccb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:47.724100 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94992ce4-b602-4c57-a299-36eb9496bccb None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:47.734691 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 457/926] 10.222.0.39 () {56 vars in 1126 bytes} [Fri Aug 5 19:50:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:47.735805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94992ce4-b602-4c57-a299-36eb9496bccb None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:47.790200 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:47.804547 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5db44a-4adc-4531-8095-a78954a9b868 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:47.813994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:47.847117 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894918be-d7f3-450f-8405-10dd2b9b02df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:47.848243 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894918be-d7f3-450f-8405-10dd2b9b02df None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:47.848817 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894918be-d7f3-450f-8405-10dd2b9b02df None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:47.849301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894918be-d7f3-450f-8405-10dd2b9b02df None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:47.850207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894918be-d7f3-450f-8405-10dd2b9b02df None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:47.850760 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894918be-d7f3-450f-8405-10dd2b9b02df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:47.852409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894918be-d7f3-450f-8405-10dd2b9b02df None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:47.863681 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 470/927] 10.222.0.39 () {56 vars in 1122 bytes} [Fri Aug 5 19:50:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:47.864954 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894918be-d7f3-450f-8405-10dd2b9b02df None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:47.865233 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94992ce4-b602-4c57-a299-36eb9496bccb None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:47.904891 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d66f9c-b68e-4084-9bd3-b0a6ce33cdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:47.905736 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d66f9c-b68e-4084-9bd3-b0a6ce33cdbd None admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:47.906088 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d66f9c-b68e-4084-9bd3-b0a6ce33cdbd None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:47.906518 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d66f9c-b68e-4084-9bd3-b0a6ce33cdbd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:47.907711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d66f9c-b68e-4084-9bd3-b0a6ce33cdbd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:47.908927 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d66f9c-b68e-4084-9bd3-b0a6ce33cdbd None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:47.948609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 458/928] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:47 2022] POST /identity/v3/endpoints => generated 347 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:47.999751 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894918be-d7f3-450f-8405-10dd2b9b02df None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:48.052772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ca096b-42b5-4174-a51f-9d937f72b98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:48.063280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca096b-42b5-4174-a51f-9d937f72b98a None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:48.063843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca096b-42b5-4174-a51f-9d937f72b98a None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:48.064134 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca096b-42b5-4174-a51f-9d937f72b98a None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:48.064923 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ca096b-42b5-4174-a51f-9d937f72b98a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:48.065234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ca096b-42b5-4174-a51f-9d937f72b98a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:48.066940 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ca096b-42b5-4174-a51f-9d937f72b98a None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:48.078233 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 471/929] 10.222.0.39 () {56 vars in 1122 bytes} [Fri Aug 5 19:50:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:48.084862 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d66f9c-b68e-4084-9bd3-b0a6ce33cdbd None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:48.231755 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d66f9c-b68e-4084-9bd3-b0a6ce33cdbd None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:48.246707 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:48.272237 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680706e7-c2ed-41a6-bc6d-ebf4772d53f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:48.272825 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680706e7-c2ed-41a6-bc6d-ebf4772d53f4 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:48.273365 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680706e7-c2ed-41a6-bc6d-ebf4772d53f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:48.273640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680706e7-c2ed-41a6-bc6d-ebf4772d53f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:48.274462 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680706e7-c2ed-41a6-bc6d-ebf4772d53f4 None admin] RBAC: Adding query filter params (interface=public, service_id=f771c5d061c24ba9b68d17cb90141708, region_id=RegionOne) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:48.274821 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680706e7-c2ed-41a6-bc6d-ebf4772d53f4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:48.276265 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680706e7-c2ed-41a6-bc6d-ebf4772d53f4 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:48.282281 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 459/930] 10.222.0.39 () {56 vars in 1251 bytes} [Fri Aug 5 19:50:48 2022] GET /identity/v3/endpoints?service_id=f771c5d061c24ba9b68d17cb90141708&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:48.288007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ca096b-42b5-4174-a51f-9d937f72b98a None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:48.305976 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:48.409091 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ca096b-42b5-4174-a51f-9d937f72b98a None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:48.450940 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81cb772-48c6-4aa2-ac26-2d2083faf268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:48.451827 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81cb772-48c6-4aa2-ac26-2d2083faf268 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:48.452254 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81cb772-48c6-4aa2-ac26-2d2083faf268 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:48.452706 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81cb772-48c6-4aa2-ac26-2d2083faf268 None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:48.453529 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81cb772-48c6-4aa2-ac26-2d2083faf268 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:48.455171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81cb772-48c6-4aa2-ac26-2d2083faf268 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:48.463686 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 472/931] 10.222.0.39 () {56 vars in 1087 bytes} [Fri Aug 5 19:50:48 2022] GET /identity/v3/services => generated 1497 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:48.987854 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc1196-23bd-4ea7-a7dd-a99671fe54a0 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:48.988422 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc1196-23bd-4ea7-a7dd-a99671fe54a0 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:48.988422 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc1196-23bd-4ea7-a7dd-a99671fe54a0 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:48.990234 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 460/932] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:48.996762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b018d0e-67d1-4772-9c84-c9cd9a4d336a None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:48.997172 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b018d0e-67d1-4772-9c84-c9cd9a4d336a None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:48.997459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b018d0e-67d1-4772-9c84-c9cd9a4d336a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:49.016855 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-2b018d0e-67d1-4772-9c84-c9cd9a4d336a None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:49.018084 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-2b018d0e-67d1-4772-9c84-c9cd9a4d336a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:49.097435 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 473/933] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:49.121640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc1196-23bd-4ea7-a7dd-a99671fe54a0 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:49.133206 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a630f8-d490-4ae5-b222-71e8f00323ce None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:49.133483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a630f8-d490-4ae5-b222-71e8f00323ce None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:49.133760 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a630f8-d490-4ae5-b222-71e8f00323ce None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:49.136289 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 474/934] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:49.142698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c2dcb-8525-492f-801c-e1d056c81337 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:49.142698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c2dcb-8525-492f-801c-e1d056c81337 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:49.143160 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c2dcb-8525-492f-801c-e1d056c81337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:49.165571 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-015c2dcb-8525-492f-801c-e1d056c81337 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:49.167032 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-015c2dcb-8525-492f-801c-e1d056c81337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:49.175497 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:49.249014 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc1196-23bd-4ea7-a7dd-a99671fe54a0 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:49.249310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 475/935] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:49.272247 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c2dcb-8525-492f-801c-e1d056c81337 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:49.304227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09113f-513d-4270-b29b-b8e465755c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:49.307734 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09113f-513d-4270-b29b-b8e465755c05 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:49.308732 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09113f-513d-4270-b29b-b8e465755c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:49.309666 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09113f-513d-4270-b29b-b8e465755c05 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:49.325482 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b09113f-513d-4270-b29b-b8e465755c05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:49.332827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-8b09113f-513d-4270-b29b-b8e465755c05 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:50:49.332827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:49.332827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:49.332827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:49.332827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:49.332827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:49.332827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:49.332827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:49.332827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:49.332827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:49.332827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:49.332827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:49.332827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:49.332827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:49.332827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:49.332827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:49.333796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:49.333796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:49.333796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:49.333796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:49.333796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:49.333796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:49.333796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:49.333796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:49.333796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:49.333796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:49.333796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:49.333796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:49.333796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:49.333796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:49.333796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:49.333796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:49.334630 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:49.334630 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:49.334630 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:49.334630 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:49.334630 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:49.334630 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:49.334630 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:49.334630 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:49.334630 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:49.334630 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:49.334630 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:50:49.334630 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:49.343280 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 461/936] 10.222.0.39 () {56 vars in 1120 bytes} [Fri Aug 5 19:50:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:49.344829 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09113f-513d-4270-b29b-b8e465755c05 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:49.387449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:49.401128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c2dcb-8525-492f-801c-e1d056c81337 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:49.443891 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ebbe3-fdd8-478b-b06f-fb1e3ab5d0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:49.445156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ebbe3-fdd8-478b-b06f-fb1e3ab5d0ba None admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:49.445933 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ebbe3-fdd8-478b-b06f-fb1e3ab5d0ba None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:49.446465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ebbe3-fdd8-478b-b06f-fb1e3ab5d0ba None admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:49.447694 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425ebbe3-fdd8-478b-b06f-fb1e3ab5d0ba None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:49.449541 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425ebbe3-fdd8-478b-b06f-fb1e3ab5d0ba None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:49.452509 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: WARNING keystone.server.flask.common [None req-425ebbe3-fdd8-478b-b06f-fb1e3ab5d0ba None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 05 19:50:49.462186 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:49.468809 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09113f-513d-4270-b29b-b8e465755c05 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:49.474784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.sql.core [None req-425ebbe3-fdd8-478b-b06f-fb1e3ab5d0ba None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 05 19:50:49.474784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 05 19:50:49.474784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [parameters: {'id': 'a78f9e32efb440efa6dc7869d9ac602e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 05 19:50:49.474784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80478) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 05 19:50:49.495099 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-425ebbe3-fdd8-478b-b06f-fb1e3ab5d0ba None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a78f9e32efb440efa6dc7869d9ac602e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a78f9e32efb440efa6dc7869d9ac602e. Aug 05 19:50:49.495099 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:49.495099 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 19:50:49.495099 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 19:50:49.495099 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 19:50:49.495099 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 19:50:49.495099 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 19:50:49.495099 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 19:50:49.495099 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 19:50:49.495099 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application conn.query(q) Aug 05 19:50:49.495099 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 19:50:49.495099 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 19:50:49.495099 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 19:50:49.496139 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application result.read() Aug 05 19:50:49.496139 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 19:50:49.496139 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 19:50:49.496139 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 19:50:49.496139 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 19:50:49.496139 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 19:50:49.496139 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 19:50:49.496139 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 19:50:49.496139 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 19:50:49.496139 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 05 19:50:49.496139 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:49.496139 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 05 19:50:49.496139 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:49.496139 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:49.496139 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 05 19:50:49.496139 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 05 19:50:49.496139 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 05 19:50:49.496139 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 05 19:50:49.497128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 19:50:49.497128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application next(self.gen) Aug 05 19:50:49.497128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 05 19:50:49.497128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application yield resource Aug 05 19:50:49.497128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 19:50:49.497128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application next(self.gen) Aug 05 19:50:49.497128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 05 19:50:49.497128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self.session.rollback() Aug 05 19:50:49.497128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 05 19:50:49.497128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self.force_reraise() Aug 05 19:50:49.497128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 05 19:50:49.497128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise self.value Aug 05 19:50:49.497128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 05 19:50:49.497128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 05 19:50:49.497128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 05 19:50:49.497128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application session.commit() Aug 05 19:50:49.497128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 05 19:50:49.498029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 05 19:50:49.498029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 05 19:50:49.498029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self._prepare_impl() Aug 05 19:50:49.498029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 05 19:50:49.498029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self.session.flush() Aug 05 19:50:49.498029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 05 19:50:49.498029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self._flush(objects) Aug 05 19:50:49.498029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 05 19:50:49.498029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 05 19:50:49.498029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 05 19:50:49.498029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application compat.raise_( Aug 05 19:50:49.498029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 19:50:49.498029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception Aug 05 19:50:49.498029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 05 19:50:49.498029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application flush_context.execute() Aug 05 19:50:49.498029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 05 19:50:49.498029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rec.execute(self) Aug 05 19:50:49.498919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 05 19:50:49.498919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 05 19:50:49.498919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 05 19:50:49.498919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 05 19:50:49.498919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 05 19:50:49.498919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 05 19:50:49.498919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 05 19:50:49.498919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 05 19:50:49.498919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 05 19:50:49.498919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 05 19:50:49.498919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 05 19:50:49.498919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 05 19:50:49.498919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 05 19:50:49.498919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 05 19:50:49.498919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 05 19:50:49.499765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 05 19:50:49.499765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 19:50:49.499765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception Aug 05 19:50:49.499765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 19:50:49.499765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 19:50:49.499765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 19:50:49.499765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 19:50:49.499765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 19:50:49.499765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 19:50:49.499765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 19:50:49.499765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application conn.query(q) Aug 05 19:50:49.499765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 19:50:49.499765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 19:50:49.499765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 19:50:49.499765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application result.read() Aug 05 19:50:49.499765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 19:50:49.499765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 19:50:49.501848 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 19:50:49.501848 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 19:50:49.501848 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 19:50:49.501848 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 19:50:49.501848 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 19:50:49.501848 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 19:50:49.501848 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 05 19:50:49.501848 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 05 19:50:49.501848 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [parameters: {'id': 'a78f9e32efb440efa6dc7869d9ac602e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 05 19:50:49.501848 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 05 19:50:49.501848 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:49.501848 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:49.501848 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:49.501848 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:49.501848 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 05 19:50:49.503695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 05 19:50:49.503695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 05 19:50:49.503695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 05 19:50:49.503695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 05 19:50:49.503695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:49.503695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:49.503695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:49.503695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:49.503695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:49.503695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:49.503695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:49.503695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:49.503695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:49.503695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:49.503695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:49.503695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:49.503695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:49.505922 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:49.505922 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 05 19:50:49.505922 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 05 19:50:49.505922 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:49.505922 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:49.505922 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 05 19:50:49.505922 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 05 19:50:49.505922 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a78f9e32efb440efa6dc7869d9ac602e. Aug 05 19:50:49.505922 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:49.512782 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 476/937] 10.222.0.39 () {60 vars in 1140 bytes} [Fri Aug 5 19:50:49 2022] POST /identity/v3/projects => generated 262 bytes in 235 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:49.512946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d7715-95d9-45e8-90d3-8ae60d9a98c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:49.512946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d7715-95d9-45e8-90d3-8ae60d9a98c3 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:49.512946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d7715-95d9-45e8-90d3-8ae60d9a98c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:49.512946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d7715-95d9-45e8-90d3-8ae60d9a98c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:49.513525 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ebbe3-fdd8-478b-b06f-fb1e3ab5d0ba None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:49.514538 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259d7715-95d9-45e8-90d3-8ae60d9a98c3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:49.516796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259d7715-95d9-45e8-90d3-8ae60d9a98c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:49.518211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259d7715-95d9-45e8-90d3-8ae60d9a98c3 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:49.536469 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 462/938] 10.222.0.39 () {56 vars in 1115 bytes} [Fri Aug 5 19:50:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:49.537754 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d7715-95d9-45e8-90d3-8ae60d9a98c3 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:49.654820 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ebbe3-fdd8-478b-b06f-fb1e3ab5d0ba None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:49.670402 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:49.676335 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d7715-95d9-45e8-90d3-8ae60d9a98c3 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:49.703476 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf746884-4595-4a3f-b317-25fa73ca0fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:49.704143 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf746884-4595-4a3f-b317-25fa73ca0fdf None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:49.704425 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf746884-4595-4a3f-b317-25fa73ca0fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:49.704743 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf746884-4595-4a3f-b317-25fa73ca0fdf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:49.715214 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf746884-4595-4a3f-b317-25fa73ca0fdf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:49.717322 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf746884-4595-4a3f-b317-25fa73ca0fdf None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:49.718168 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d57670c-6187-4313-9f10-892498eb3949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:49.719069 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57670c-6187-4313-9f10-892498eb3949 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:49.719544 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57670c-6187-4313-9f10-892498eb3949 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:49.720010 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57670c-6187-4313-9f10-892498eb3949 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:49.732429 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d57670c-6187-4313-9f10-892498eb3949 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:49.732649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-cf746884-4595-4a3f-b317-25fa73ca0fdf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:49.732649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:49.732649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:49.732649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:49.732649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:49.732649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:49.732649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:49.732649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:49.732649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:49.732649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:49.732649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:49.732649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:49.732649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:49.732649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:49.732649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:49.733508 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:49.733508 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:49.733508 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:49.733508 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:49.733508 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:49.733508 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:49.733508 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:49.733508 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:49.733508 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:49.733508 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:49.733508 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:49.733508 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:49.733508 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:49.733508 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:49.733508 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:49.733508 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:49.733508 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:49.734383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:49.734383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:49.734383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:49.734383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:49.734383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:49.734383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:49.734383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:49.734383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:49.736822 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d57670c-6187-4313-9f10-892498eb3949 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:49.738581 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 477/939] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:49.739932 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf746884-4595-4a3f-b317-25fa73ca0fdf None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:49.756475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-8d57670c-6187-4313-9f10-892498eb3949 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:49.756475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:49.756475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:49.756475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:49.756475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:49.756475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:49.756475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:49.756475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:49.756475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:49.756475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:49.756475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:49.756475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:49.756475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:49.756475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:49.756475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:49.757461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:49.757461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:49.757461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:49.757461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:49.757461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:49.757461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:49.757461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:49.757461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:49.757461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:49.757461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:49.757461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:49.757461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:49.757461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:49.757461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:49.757461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:49.757461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:49.757461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:49.758998 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:49.758998 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:49.758998 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:49.758998 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:49.763294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 463/940] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:49.764315 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d57670c-6187-4313-9f10-892498eb3949 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:49.825656 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:49.883902 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf746884-4595-4a3f-b317-25fa73ca0fdf None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:49.895018 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d57670c-6187-4313-9f10-892498eb3949 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:49.924494 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a61b32-cd45-4e0c-8af3-2c9df4557186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:49.925502 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a61b32-cd45-4e0c-8af3-2c9df4557186 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:49.925998 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a61b32-cd45-4e0c-8af3-2c9df4557186 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:49.926474 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a61b32-cd45-4e0c-8af3-2c9df4557186 None admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:49.927418 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a61b32-cd45-4e0c-8af3-2c9df4557186 None admin] RBAC: Adding query filter params (name=service) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:49.927931 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a61b32-cd45-4e0c-8af3-2c9df4557186 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:49.929874 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a61b32-cd45-4e0c-8af3-2c9df4557186 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:49.933112 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e45e0-86e6-4c9b-b612-64f20e0b0886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:49.933890 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e45e0-86e6-4c9b-b612-64f20e0b0886 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:49.934191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e45e0-86e6-4c9b-b612-64f20e0b0886 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:49.934464 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e45e0-86e6-4c9b-b612-64f20e0b0886 None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:49.935165 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87e45e0-86e6-4c9b-b612-64f20e0b0886 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:49.935950 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87e45e0-86e6-4c9b-b612-64f20e0b0886 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:49.937567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87e45e0-86e6-4c9b-b612-64f20e0b0886 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:49.956216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 464/941] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:49.958134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e45e0-86e6-4c9b-b612-64f20e0b0886 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:49.964741 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 478/942] 10.222.0.39 () {56 vars in 1112 bytes} [Fri Aug 5 19:50:49 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:49.973846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:50.093818 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e45e0-86e6-4c9b-b612-64f20e0b0886 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:50.139830 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5996161-bef6-462a-9448-7eee2a72ad22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:50.140496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5996161-bef6-462a-9448-7eee2a72ad22 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:50.140809 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5996161-bef6-462a-9448-7eee2a72ad22 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:50.144006 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5996161-bef6-462a-9448-7eee2a72ad22 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:50.153802 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5996161-bef6-462a-9448-7eee2a72ad22 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:50.155263 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5996161-bef6-462a-9448-7eee2a72ad22 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:50.171108 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-c5996161-bef6-462a-9448-7eee2a72ad22 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:50:50.171108 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:50.171108 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:50.171108 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:50.171108 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:50.171108 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:50.171108 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:50.171108 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:50.171108 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:50.171108 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:50.171108 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:50.171108 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:50.171108 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:50.171108 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:50.171108 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:50.171108 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:50.172131 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:50.172131 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:50.172131 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:50.172131 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:50.172131 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:50.172131 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:50.172131 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:50.172131 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:50.172131 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:50.172131 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:50.172131 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:50.172131 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:50.172131 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:50.172131 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:50.172131 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:50.172131 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:50.172131 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:50.172131 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:50.173129 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:50.173129 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:50.173129 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:50.173129 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:50.173129 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:50.173129 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:50.173129 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:50.173129 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:50.173129 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:50:50.173129 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:50.173129 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 465/943] 10.222.0.39 () {56 vars in 1117 bytes} [Fri Aug 5 19:50:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:50.179724 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a61b32-cd45-4e0c-8af3-2c9df4557186 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:50.300927 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a61b32-cd45-4e0c-8af3-2c9df4557186 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:50.320973 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:50.341345 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80646169-f8b6-4bcb-8f40-4a0e461bf1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:50.341981 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80646169-f8b6-4bcb-8f40-4a0e461bf1ff None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:50.342259 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80646169-f8b6-4bcb-8f40-4a0e461bf1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:50.342529 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80646169-f8b6-4bcb-8f40-4a0e461bf1ff None admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:50.343354 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80646169-f8b6-4bcb-8f40-4a0e461bf1ff None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:50.343656 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80646169-f8b6-4bcb-8f40-4a0e461bf1ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:50.345316 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80646169-f8b6-4bcb-8f40-4a0e461bf1ff None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:50.359742 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ae043-5d5b-4aa3-a32a-2eed4988c7a4 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:50.360218 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ae043-5d5b-4aa3-a32a-2eed4988c7a4 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:50.360834 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ae043-5d5b-4aa3-a32a-2eed4988c7a4 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:50.363393 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 466/944] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:50:50.367292 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 479/945] 10.222.0.39 () {56 vars in 1149 bytes} [Fri Aug 5 19:50:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:50.371218 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ca1a5-5ca9-4d83-a4ff-d480fa3808b4 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:50.371868 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ca1a5-5ca9-4d83-a4ff-d480fa3808b4 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:50.371978 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ca1a5-5ca9-4d83-a4ff-d480fa3808b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:50.372896 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80646169-f8b6-4bcb-8f40-4a0e461bf1ff None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:50.397673 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-1b7ca1a5-5ca9-4d83-a4ff-d480fa3808b4 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:50.399135 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-1b7ca1a5-5ca9-4d83-a4ff-d480fa3808b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:50.488615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:50.500937 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 467/946] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:50.516339 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80646169-f8b6-4bcb-8f40-4a0e461bf1ff None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:50.523971 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ca1a5-5ca9-4d83-a4ff-d480fa3808b4 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:50.563058 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618acfe-b920-4fdf-b4ec-948a450432a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:50.564069 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618acfe-b920-4fdf-b4ec-948a450432a6 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:50.564550 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618acfe-b920-4fdf-b4ec-948a450432a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:50.565275 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618acfe-b920-4fdf-b4ec-948a450432a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:50.580314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5618acfe-b920-4fdf-b4ec-948a450432a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:50.582550 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5618acfe-b920-4fdf-b4ec-948a450432a6 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:50.603035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-5618acfe-b920-4fdf-b4ec-948a450432a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:50.603035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:50.603035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:50.603035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:50.603035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:50.603035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:50.603035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:50.603035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:50.603035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:50.603035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:50.603035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:50.603035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:50.603035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:50.603035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:50.603035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:50.604106 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:50.604106 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:50.604106 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:50.604106 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:50.604106 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:50.604106 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:50.604106 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:50.604106 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:50.604106 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:50.604106 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:50.604106 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:50.604106 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:50.604106 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:50.604106 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:50.604106 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:50.604106 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:50.604106 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:50.605109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:50.605109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:50.605109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:50.605109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:50.608765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 480/947] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:50.611206 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39d224-566e-4220-9289-00a9a4904b4b None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:50.611633 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39d224-566e-4220-9289-00a9a4904b4b None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:50.612094 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39d224-566e-4220-9289-00a9a4904b4b None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:50.613736 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 481/948] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:50.618796 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39d224-566e-4220-9289-00a9a4904b4b None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:50.648221 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:50.659478 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ca1a5-5ca9-4d83-a4ff-d480fa3808b4 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:50.703292 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363188a-3f55-4292-9845-996a379ceefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:50.703969 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363188a-3f55-4292-9845-996a379ceefd None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:50.704249 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363188a-3f55-4292-9845-996a379ceefd None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:50.704566 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363188a-3f55-4292-9845-996a379ceefd None admin] PATH_INFO: `/v3/services/compute` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:50.705231 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5363188a-3f55-4292-9845-996a379ceefd None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:50.706613 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5363188a-3f55-4292-9845-996a379ceefd None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:50.716916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-5363188a-3f55-4292-9845-996a379ceefd None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 19:50:50.716916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:50.716916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:50.716916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:50.716916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:50.716916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:50.716916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:50.716916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:50.716916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 19:50:50.716916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:50.716916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:50.716916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:50.716916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:50.716916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:50.716916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:50.716916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:50.716916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:50.718017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:50.718017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:50.718017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:50.718017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:50.718017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:50.718017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:50.718017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:50.718017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:50.718017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:50.718017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:50.718017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:50.718017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:50.718017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:50.718017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:50.718017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:50.718017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:50.718876 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:50.718876 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:50.718876 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:50.718876 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:50.718876 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:50.718876 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:50.718876 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:50.718876 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:50.718876 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:50.718876 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:50.718876 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:50.718876 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:50.718876 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 19:50:50.718876 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:50.718876 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 468/949] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:50.719678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279c5fb-0204-4523-b0d6-12489d217050 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:50.719678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279c5fb-0204-4523-b0d6-12489d217050 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:50.720597 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279c5fb-0204-4523-b0d6-12489d217050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:50.740543 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-4279c5fb-0204-4523-b0d6-12489d217050 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:50.741822 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-4279c5fb-0204-4523-b0d6-12489d217050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:50.755990 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39d224-566e-4220-9289-00a9a4904b4b None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:50.793228 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae7afa-685e-4c68-ab35-9017088ca6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:50.793855 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae7afa-685e-4c68-ab35-9017088ca6b1 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:50.794129 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae7afa-685e-4c68-ab35-9017088ca6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:50.794411 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae7afa-685e-4c68-ab35-9017088ca6b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:50.795115 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae7afa-685e-4c68-ab35-9017088ca6b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:50.795422 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae7afa-685e-4c68-ab35-9017088ca6b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:50.796844 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae7afa-685e-4c68-ab35-9017088ca6b1 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:50.815528 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 482/950] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:50.815528 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae7afa-685e-4c68-ab35-9017088ca6b1 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:50.819848 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:50.836706 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 469/951] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:50.838423 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279c5fb-0204-4523-b0d6-12489d217050 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:50.914075 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:50.933056 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae7afa-685e-4c68-ab35-9017088ca6b1 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:50.963833 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279c5fb-0204-4523-b0d6-12489d217050 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:50.974716 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99279ce5-fba4-4ed4-a084-1aa4fac52290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:50.975349 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99279ce5-fba4-4ed4-a084-1aa4fac52290 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:50.975922 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99279ce5-fba4-4ed4-a084-1aa4fac52290 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:50.975922 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99279ce5-fba4-4ed4-a084-1aa4fac52290 None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:50.976643 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99279ce5-fba4-4ed4-a084-1aa4fac52290 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:50.976954 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99279ce5-fba4-4ed4-a084-1aa4fac52290 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:50.978347 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99279ce5-fba4-4ed4-a084-1aa4fac52290 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:50.989102 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 483/952] 10.222.0.39 () {56 vars in 1112 bytes} [Fri Aug 5 19:50:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:50.989102 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99279ce5-fba4-4ed4-a084-1aa4fac52290 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:51.010003 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932bc94-edb8-4dfc-9c5d-8525491e8308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:51.011017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932bc94-edb8-4dfc-9c5d-8525491e8308 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:51.011506 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932bc94-edb8-4dfc-9c5d-8525491e8308 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:51.011899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932bc94-edb8-4dfc-9c5d-8525491e8308 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:51.020000 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9932bc94-edb8-4dfc-9c5d-8525491e8308 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:51.021626 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9932bc94-edb8-4dfc-9c5d-8525491e8308 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:51.035409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-9932bc94-edb8-4dfc-9c5d-8525491e8308 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:51.035409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:51.035409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:51.035409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:51.035409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:51.035409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:51.035409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:51.035409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:51.035409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:51.035409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:51.035409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:51.035409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:51.035409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:51.035409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:51.035409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:51.036400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:51.036400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:51.036400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:51.036400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:51.036400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:51.036400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:51.036400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:51.036400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:51.036400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:51.036400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:51.036400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:51.036400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:51.036400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:51.036400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:51.036400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:51.036400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:51.036400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:51.038194 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:51.038194 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:51.038194 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:51.038194 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:51.038194 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:51.038194 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:51.038194 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:51.038194 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:51.040963 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 470/953] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:51.043468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932bc94-edb8-4dfc-9c5d-8525491e8308 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:51.111040 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99279ce5-fba4-4ed4-a084-1aa4fac52290 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:51.117368 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:51.157885 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97863747-6195-41fd-9a5a-2e8d731c9353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:51.158860 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97863747-6195-41fd-9a5a-2e8d731c9353 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:51.159357 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97863747-6195-41fd-9a5a-2e8d731c9353 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:51.159930 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97863747-6195-41fd-9a5a-2e8d731c9353 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:51.160568 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97863747-6195-41fd-9a5a-2e8d731c9353 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:51.161984 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97863747-6195-41fd-9a5a-2e8d731c9353 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:51.173719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932bc94-edb8-4dfc-9c5d-8525491e8308 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:51.173912 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-97863747-6195-41fd-9a5a-2e8d731c9353 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:50:51.173912 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:51.173912 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:51.173912 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:51.173912 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:51.173912 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:51.173912 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:51.173912 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:51.173912 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:50:51.173912 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:51.173912 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:51.173912 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:51.173912 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:51.173912 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:51.173912 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:51.173912 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:51.173912 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:51.174949 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:51.174949 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:51.174949 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:51.174949 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:51.174949 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:51.174949 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:51.174949 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:51.174949 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:51.174949 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:51.174949 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:51.174949 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:51.174949 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:51.174949 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:51.174949 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:51.174949 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:51.174949 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:51.175886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:51.175886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:51.175886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:51.175886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:51.175886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:51.175886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:51.175886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:51.175886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:51.175886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:51.175886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:51.175886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:51.175886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:51.175886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:50:51.175886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:51.176985 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 484/954] 10.222.0.39 () {56 vars in 1126 bytes} [Fri Aug 5 19:50:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:51.177630 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97863747-6195-41fd-9a5a-2e8d731c9353 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:51.197279 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:51.224035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c6143-4eeb-4bb8-8ec6-0c314f75e46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:51.224682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c6143-4eeb-4bb8-8ec6-0c314f75e46c None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:51.224962 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c6143-4eeb-4bb8-8ec6-0c314f75e46c None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:51.225236 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c6143-4eeb-4bb8-8ec6-0c314f75e46c None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:51.225918 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c6143-4eeb-4bb8-8ec6-0c314f75e46c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:51.226225 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c6143-4eeb-4bb8-8ec6-0c314f75e46c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:51.227593 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c6143-4eeb-4bb8-8ec6-0c314f75e46c None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:51.233670 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 471/955] 10.222.0.39 () {56 vars in 1112 bytes} [Fri Aug 5 19:50:51 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:51.234504 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c6143-4eeb-4bb8-8ec6-0c314f75e46c None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:51.352411 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97863747-6195-41fd-9a5a-2e8d731c9353 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:51.407799 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c6143-4eeb-4bb8-8ec6-0c314f75e46c None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:51.415397 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43433d-3e61-4999-8a10-ecd5e699670e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:51.417872 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43433d-3e61-4999-8a10-ecd5e699670e None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:51.418349 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43433d-3e61-4999-8a10-ecd5e699670e None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:51.418841 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43433d-3e61-4999-8a10-ecd5e699670e None admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:51.420624 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da43433d-3e61-4999-8a10-ecd5e699670e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:51.420766 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da43433d-3e61-4999-8a10-ecd5e699670e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:51.422163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da43433d-3e61-4999-8a10-ecd5e699670e None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:51.433668 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:51.441086 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 485/956] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:51.448673 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43433d-3e61-4999-8a10-ecd5e699670e None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:51.461429 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f4344b-3aab-4fbb-bc47-eb00d3fa172c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:51.462494 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4344b-3aab-4fbb-bc47-eb00d3fa172c None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:51.462860 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4344b-3aab-4fbb-bc47-eb00d3fa172c None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:51.463143 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4344b-3aab-4fbb-bc47-eb00d3fa172c None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:51.463839 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f4344b-3aab-4fbb-bc47-eb00d3fa172c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:51.466056 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f4344b-3aab-4fbb-bc47-eb00d3fa172c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:51.467565 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f4344b-3aab-4fbb-bc47-eb00d3fa172c None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:51.476552 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 472/957] 10.222.0.39 () {56 vars in 1122 bytes} [Fri Aug 5 19:50:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:51.484004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f4344b-3aab-4fbb-bc47-eb00d3fa172c None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:51.540595 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:51.578256 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43433d-3e61-4999-8a10-ecd5e699670e None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:51.597566 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f4344b-3aab-4fbb-bc47-eb00d3fa172c None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:51.621244 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b14304-00fb-48a3-9c0c-a66e16c0df3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:51.622580 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b14304-00fb-48a3-9c0c-a66e16c0df3c None admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:51.623082 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b14304-00fb-48a3-9c0c-a66e16c0df3c None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:51.623539 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b14304-00fb-48a3-9c0c-a66e16c0df3c None admin] PATH_INFO: `/v3/projects/80379e2104644e98a2e85d4e4af1b365/users/3488a7d2258242c0859a105c1d162767/roles/0a40666ef2c140d1951dea50f3127507` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:51.626418 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b14304-00fb-48a3-9c0c-a66e16c0df3c None admin] RBAC: Authorizing `identity:create_grant(project_id=80379e2104644e98a2e85d4e4af1b365, user_id=3488a7d2258242c0859a105c1d162767, role_id=0a40666ef2c140d1951dea50f3127507)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:51.628181 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b14304-00fb-48a3-9c0c-a66e16c0df3c None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:51.639426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af020a-aa7e-4544-8e20-c752a84a41bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:51.640309 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af020a-aa7e-4544-8e20-c752a84a41bd None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:51.640608 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af020a-aa7e-4544-8e20-c752a84a41bd None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:51.640945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af020a-aa7e-4544-8e20-c752a84a41bd None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:51.642090 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af020a-aa7e-4544-8e20-c752a84a41bd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:51.642486 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af020a-aa7e-4544-8e20-c752a84a41bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:51.643926 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af020a-aa7e-4544-8e20-c752a84a41bd None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:51.649473 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 486/958] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Aug 5 19:50:51 2022] PUT /identity/v3/projects/80379e2104644e98a2e85d4e4af1b365/users/3488a7d2258242c0859a105c1d162767/roles/0a40666ef2c140d1951dea50f3127507 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:50:51.652620 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 473/959] 10.222.0.39 () {56 vars in 1122 bytes} [Fri Aug 5 19:50:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:51.657173 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b14304-00fb-48a3-9c0c-a66e16c0df3c None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:51.795583 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b14304-00fb-48a3-9c0c-a66e16c0df3c None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:51.832156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:51.839637 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db6e7b7-1010-45f2-a060-077e204cc83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:51.840373 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6e7b7-1010-45f2-a060-077e204cc83a None admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:51.840762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6e7b7-1010-45f2-a060-077e204cc83a None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:51.842791 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6e7b7-1010-45f2-a060-077e204cc83a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:51.843636 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db6e7b7-1010-45f2-a060-077e204cc83a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:51.845586 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db6e7b7-1010-45f2-a060-077e204cc83a None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:51.870881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 487/960] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:51 2022] POST /identity/v3/endpoints => generated 349 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:50:52.475751 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e2224-4182-4b6b-8664-f52c16ddda94 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:52.476265 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e2224-4182-4b6b-8664-f52c16ddda94 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:52.476265 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e2224-4182-4b6b-8664-f52c16ddda94 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:52.477808 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 474/961] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:52.483646 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f465b-4f5f-4d60-912b-8ee8640bd7aa None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:52.483937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f465b-4f5f-4d60-912b-8ee8640bd7aa None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:52.484210 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f465b-4f5f-4d60-912b-8ee8640bd7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:52.509410 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-fc7f465b-4f5f-4d60-912b-8ee8640bd7aa None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:52.512087 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-fc7f465b-4f5f-4d60-912b-8ee8640bd7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:52.603663 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 488/962] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:52.632743 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e2224-4182-4b6b-8664-f52c16ddda94 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:52.692002 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:52.754590 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e2224-4182-4b6b-8664-f52c16ddda94 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:52.791199 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ab739-46cc-4df4-a373-b159c807f0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:52.792192 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ab739-46cc-4df4-a373-b159c807f0c3 None admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:52.792477 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ab739-46cc-4df4-a373-b159c807f0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:52.792785 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ab739-46cc-4df4-a373-b159c807f0c3 None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:52.793535 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5ab739-46cc-4df4-a373-b159c807f0c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:52.795008 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5ab739-46cc-4df4-a373-b159c807f0c3 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:52.797581 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: WARNING keystone.server.flask.common [None req-ed5ab739-46cc-4df4-a373-b159c807f0c3 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 05 19:50:52.810103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.sql.core [None req-ed5ab739-46cc-4df4-a373-b159c807f0c3 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 05 19:50:52.810103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 05 19:50:52.810103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [parameters: {'user_id': '0b31cdc3d1454056a8373de25f355e67', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 05 19:50:52.810103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80477) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 05 19:50:52.817267 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-ed5ab739-46cc-4df4-a373-b159c807f0c3 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 05 19:50:52.817267 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:52.817267 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 19:50:52.817267 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 19:50:52.817267 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 19:50:52.817267 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 19:50:52.817267 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 19:50:52.817267 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 19:50:52.817267 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 19:50:52.817267 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application conn.query(q) Aug 05 19:50:52.817267 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 19:50:52.817267 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 19:50:52.817267 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 19:50:52.817267 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application result.read() Aug 05 19:50:52.817267 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 19:50:52.818134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 19:50:52.818134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 19:50:52.818134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 19:50:52.818134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 19:50:52.818134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 19:50:52.818134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 19:50:52.818134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 19:50:52.818134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 05 19:50:52.818134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:52.818134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 05 19:50:52.818134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:52.818134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:52.818134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 05 19:50:52.818134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 05 19:50:52.818134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 05 19:50:52.818134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 05 19:50:52.818134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 19:50:52.818134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application next(self.gen) Aug 05 19:50:52.819268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 05 19:50:52.819268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application yield resource Aug 05 19:50:52.819268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 19:50:52.819268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application next(self.gen) Aug 05 19:50:52.819268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 05 19:50:52.819268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self.session.rollback() Aug 05 19:50:52.819268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 05 19:50:52.819268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self.force_reraise() Aug 05 19:50:52.819268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 05 19:50:52.819268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise self.value Aug 05 19:50:52.819268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 05 19:50:52.819268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 05 19:50:52.819268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 05 19:50:52.819268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application session.commit() Aug 05 19:50:52.819268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 05 19:50:52.819268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 05 19:50:52.819268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 05 19:50:52.820300 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self._prepare_impl() Aug 05 19:50:52.820300 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 05 19:50:52.820300 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self.session.flush() Aug 05 19:50:52.820300 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 05 19:50:52.820300 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self._flush(objects) Aug 05 19:50:52.820300 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 05 19:50:52.820300 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 05 19:50:52.820300 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 05 19:50:52.820300 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application compat.raise_( Aug 05 19:50:52.820300 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 19:50:52.820300 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception Aug 05 19:50:52.820300 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 05 19:50:52.820300 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application flush_context.execute() Aug 05 19:50:52.820300 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 05 19:50:52.820300 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rec.execute(self) Aug 05 19:50:52.820300 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 05 19:50:52.820300 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 05 19:50:52.821216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 05 19:50:52.821216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 05 19:50:52.821216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 05 19:50:52.821216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 05 19:50:52.821216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 05 19:50:52.821216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 05 19:50:52.821216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 05 19:50:52.821216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 05 19:50:52.821216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 05 19:50:52.821216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 05 19:50:52.821216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 05 19:50:52.821216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 05 19:50:52.821216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 05 19:50:52.821216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 05 19:50:52.821216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 19:50:52.821216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception Aug 05 19:50:52.822644 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 19:50:52.822644 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 19:50:52.822644 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 19:50:52.822644 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 19:50:52.822644 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 19:50:52.822644 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 19:50:52.822644 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 19:50:52.822644 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application conn.query(q) Aug 05 19:50:52.822644 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 19:50:52.822644 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 19:50:52.822644 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 19:50:52.822644 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application result.read() Aug 05 19:50:52.822644 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 19:50:52.822644 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 19:50:52.822644 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 19:50:52.822644 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 19:50:52.823551 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 19:50:52.823551 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 19:50:52.823551 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 19:50:52.823551 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 19:50:52.823551 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 05 19:50:52.823551 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 05 19:50:52.823551 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [parameters: {'user_id': '0b31cdc3d1454056a8373de25f355e67', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 05 19:50:52.823551 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 05 19:50:52.823551 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:52.823551 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:52.823551 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:52.823551 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:52.823551 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:52.823551 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:52.823551 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:52.829074 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:52.829074 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:52.829074 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:52.829074 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:52.829074 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:52.829074 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:52.829074 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:52.829074 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 05 19:50:52.829074 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 05 19:50:52.829074 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:52.829074 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:52.829074 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:52.829074 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:52.829074 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:52.829074 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:52.829074 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 05 19:50:52.829074 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 05 19:50:52.830219 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 05 19:50:52.830219 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 05 19:50:52.830219 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 05 19:50:52.830219 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 05 19:50:52.830219 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 05 19:50:52.830219 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:52.830219 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 475/963] 10.222.0.39 () {60 vars in 1131 bytes} [Fri Aug 5 19:50:52 2022] POST /identity/v3/users => generated 160 bytes in 186 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:52.830708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f465b-4f5f-4d60-912b-8ee8640bd7aa None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:52.944524 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f465b-4f5f-4d60-912b-8ee8640bd7aa None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:52.964953 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:52.983707 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca07fe-4b6e-4bf3-849a-cc6fcb216202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:52.984365 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca07fe-4b6e-4bf3-849a-cc6fcb216202 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:52.984767 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca07fe-4b6e-4bf3-849a-cc6fcb216202 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:52.985042 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca07fe-4b6e-4bf3-849a-cc6fcb216202 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:52.992935 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca07fe-4b6e-4bf3-849a-cc6fcb216202 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:52.995041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca07fe-4b6e-4bf3-849a-cc6fcb216202 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:53.007301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-32ca07fe-4b6e-4bf3-849a-cc6fcb216202 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 19:50:53.007301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:53.007301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:53.007301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:53.007301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:53.007301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:53.007301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:53.007301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:53.007301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:53.007301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:53.007301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:53.007301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:53.007301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:53.007301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:53.007301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:53.007301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:53.008217 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:53.008217 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:53.008217 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:53.008217 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:53.008217 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:53.008217 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:53.008217 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:53.008217 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:53.008217 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:53.008217 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:53.008217 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:53.008217 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:53.008217 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:53.008217 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:53.008217 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:53.008217 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:53.008217 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:53.008217 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:53.009392 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:53.009392 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:53.009392 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:53.009392 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:53.009392 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:53.009392 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:53.009392 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:53.009392 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:53.009392 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 19:50:53.009392 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:53.009392 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 489/964] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:50:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:53.016084 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ab739-46cc-4df4-a373-b159c807f0c3 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:53.154543 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ab739-46cc-4df4-a373-b159c807f0c3 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:53.165220 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:53.193331 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05d99e-05de-40f1-8bd1-fbb7b01da1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:53.194348 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05d99e-05de-40f1-8bd1-fbb7b01da1df None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:53.194837 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05d99e-05de-40f1-8bd1-fbb7b01da1df None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:53.195407 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05d99e-05de-40f1-8bd1-fbb7b01da1df None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:53.196437 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05d99e-05de-40f1-8bd1-fbb7b01da1df None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:53.196992 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05d99e-05de-40f1-8bd1-fbb7b01da1df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:53.199521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05d99e-05de-40f1-8bd1-fbb7b01da1df None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:53.240314 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 476/965] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Aug 5 19:50:53 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:53.515905 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1439e67-da29-4b9f-8cb6-532a544ba010 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:53.516299 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1439e67-da29-4b9f-8cb6-532a544ba010 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:53.516483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1439e67-da29-4b9f-8cb6-532a544ba010 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:53.518121 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 490/966] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:53.524473 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092b5d2-4050-47b0-ac05-7e2f1359f0a0 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:53.524998 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092b5d2-4050-47b0-ac05-7e2f1359f0a0 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:53.525087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092b5d2-4050-47b0-ac05-7e2f1359f0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:53.544181 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-a092b5d2-4050-47b0-ac05-7e2f1359f0a0 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:53.545724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-a092b5d2-4050-47b0-ac05-7e2f1359f0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:53.617754 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 477/967] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:53.641268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1439e67-da29-4b9f-8cb6-532a544ba010 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:53.745641 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1439e67-da29-4b9f-8cb6-532a544ba010 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:53.780566 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732171b3-bbe2-4d4f-a567-7ccbcad90d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:53.781128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732171b3-bbe2-4d4f-a567-7ccbcad90d2a None admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:53.781405 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732171b3-bbe2-4d4f-a567-7ccbcad90d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:53.781677 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732171b3-bbe2-4d4f-a567-7ccbcad90d2a None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:53.782374 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732171b3-bbe2-4d4f-a567-7ccbcad90d2a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:53.783726 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732171b3-bbe2-4d4f-a567-7ccbcad90d2a None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:53.808608 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 491/968] 10.222.0.39 () {60 vars in 1141 bytes} [Fri Aug 5 19:50:53 2022] POST /identity/v3/services => generated 242 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:53.879316 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8356685d-a549-4d82-9df2-d48fa2bf02f7 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:53.879623 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8356685d-a549-4d82-9df2-d48fa2bf02f7 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:53.879932 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8356685d-a549-4d82-9df2-d48fa2bf02f7 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:53.881579 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 478/969] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:53.888367 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd911e1-aa8b-4932-9fcc-3c6be8946cfe None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:53.888696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd911e1-aa8b-4932-9fcc-3c6be8946cfe None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:53.888961 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd911e1-aa8b-4932-9fcc-3c6be8946cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:53.909122 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-7dd911e1-aa8b-4932-9fcc-3c6be8946cfe None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:53.910335 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-7dd911e1-aa8b-4932-9fcc-3c6be8946cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:53.942563 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:53.995250 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 492/970] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:53 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:54.019078 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8356685d-a549-4d82-9df2-d48fa2bf02f7 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:54.065785 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc173d-ae9f-4c17-8176-7dac7e83a9c7 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:54.066092 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc173d-ae9f-4c17-8176-7dac7e83a9c7 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:54.066270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc173d-ae9f-4c17-8176-7dac7e83a9c7 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:54.067758 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 493/971] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:54.073680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df3ea04-98e8-41cc-9f14-42d10640ba14 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:54.073680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df3ea04-98e8-41cc-9f14-42d10640ba14 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:54.073680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df3ea04-98e8-41cc-9f14-42d10640ba14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:54.092959 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-1df3ea04-98e8-41cc-9f14-42d10640ba14 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:54.094160 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-1df3ea04-98e8-41cc-9f14-42d10640ba14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:54.130786 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:54.140886 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8356685d-a549-4d82-9df2-d48fa2bf02f7 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:54.179666 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44567f0d-7b7f-47d5-9942-addd0d2c389d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:54.180310 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44567f0d-7b7f-47d5-9942-addd0d2c389d None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:54.180617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44567f0d-7b7f-47d5-9942-addd0d2c389d None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:54.180899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44567f0d-7b7f-47d5-9942-addd0d2c389d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:54.187212 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 494/972] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:54 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:50:54.193879 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44567f0d-7b7f-47d5-9942-addd0d2c389d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:54.199968 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-44567f0d-7b7f-47d5-9942-addd0d2c389d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:50:54.199968 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:54.199968 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:54.199968 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:54.199968 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:54.199968 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:54.199968 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:54.199968 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:54.199968 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:54.199968 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:54.199968 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:54.199968 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:54.199968 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:54.199968 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:54.199968 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:54.199968 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:54.201153 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:54.201153 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:54.201153 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:54.201153 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:54.201153 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:54.201153 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:54.201153 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:54.201153 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:54.201153 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:54.201153 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:54.201153 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:54.201153 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:54.201153 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:54.201153 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:54.201153 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:54.201153 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:54.202015 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:54.202015 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:54.202015 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:54.202015 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:54.202015 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:54.202015 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:54.202015 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:54.202015 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:54.202015 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:54.202015 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:54.202015 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:50:54.202015 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:54.203277 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 479/973] 10.222.0.39 () {56 vars in 1120 bytes} [Fri Aug 5 19:50:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:54.207060 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df3ea04-98e8-41cc-9f14-42d10640ba14 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:54.210370 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44567f0d-7b7f-47d5-9942-addd0d2c389d None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:54.311036 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:54.320655 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44567f0d-7b7f-47d5-9942-addd0d2c389d None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:54.325827 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df3ea04-98e8-41cc-9f14-42d10640ba14 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:54.359915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968073ad-b541-42a5-9b10-58588ba1a3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:54.360548 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968073ad-b541-42a5-9b10-58588ba1a3de None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:54.360873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968073ad-b541-42a5-9b10-58588ba1a3de None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:54.361120 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968073ad-b541-42a5-9b10-58588ba1a3de None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:54.361775 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968073ad-b541-42a5-9b10-58588ba1a3de None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:54.363088 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968073ad-b541-42a5-9b10-58588ba1a3de None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:54.364216 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a602fd-9d37-49f5-984a-db1692f61d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:54.365037 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a602fd-9d37-49f5-984a-db1692f61d2e None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:54.365328 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a602fd-9d37-49f5-984a-db1692f61d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:54.365843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a602fd-9d37-49f5-984a-db1692f61d2e None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:54.366314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a602fd-9d37-49f5-984a-db1692f61d2e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:54.366618 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a602fd-9d37-49f5-984a-db1692f61d2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:54.367963 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a602fd-9d37-49f5-984a-db1692f61d2e None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:54.372846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-968073ad-b541-42a5-9b10-58588ba1a3de None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:50:54.372846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:54.372846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:54.372846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:54.372846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:54.372846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:54.372846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:54.372846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:54.372846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:50:54.372846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:54.372846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:54.372846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:54.372846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:54.372846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:54.372846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:54.372846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:54.372846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:54.373841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:54.373841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:54.373841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:54.373841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:54.373841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:54.373841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:54.373841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:54.373841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:54.373841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:54.373841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:54.373841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:54.373841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:54.373841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:54.373841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:54.373841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:54.373841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:54.374828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:54.374828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:54.374828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:54.374828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:54.374828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:54.374828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:54.374828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:54.374828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:54.374828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:54.374828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:54.374828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:54.374828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:54.374828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:50:54.374828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:54.374828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 480/974] 10.222.0.39 () {56 vars in 1126 bytes} [Fri Aug 5 19:50:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:54.383900 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 495/975] 10.222.0.39 () {56 vars in 1115 bytes} [Fri Aug 5 19:50:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:54.384165 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968073ad-b541-42a5-9b10-58588ba1a3de None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:54.385727 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a602fd-9d37-49f5-984a-db1692f61d2e None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:54.406578 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:54.495206 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a602fd-9d37-49f5-984a-db1692f61d2e None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:54.501402 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968073ad-b541-42a5-9b10-58588ba1a3de None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:54.532962 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bdee22-2fbb-4b62-b6f7-f05b50df2b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:54.533584 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdee22-2fbb-4b62-b6f7-f05b50df2b16 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:54.533869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdee22-2fbb-4b62-b6f7-f05b50df2b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:54.534202 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdee22-2fbb-4b62-b6f7-f05b50df2b16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:54.537332 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de600f-f656-4006-b01e-8dbe83f61b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:54.537949 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de600f-f656-4006-b01e-8dbe83f61b99 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:54.538230 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de600f-f656-4006-b01e-8dbe83f61b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:54.538503 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de600f-f656-4006-b01e-8dbe83f61b99 None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:54.539184 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51de600f-f656-4006-b01e-8dbe83f61b99 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:54.539489 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51de600f-f656-4006-b01e-8dbe83f61b99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:54.541698 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51de600f-f656-4006-b01e-8dbe83f61b99 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:54.546594 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bdee22-2fbb-4b62-b6f7-f05b50df2b16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:54.552060 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 481/976] 10.222.0.39 () {56 vars in 1122 bytes} [Fri Aug 5 19:50:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:54.552185 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bdee22-2fbb-4b62-b6f7-f05b50df2b16 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:54.552906 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de600f-f656-4006-b01e-8dbe83f61b99 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:54.567029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-54bdee22-2fbb-4b62-b6f7-f05b50df2b16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:54.567029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:54.567029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:54.567029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:54.567029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:54.567029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:54.567029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:54.567029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:54.567029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:54.567029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:54.567029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:54.567029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:54.567029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:54.567029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:54.567029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:54.568270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:54.568270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:54.568270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:54.568270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:54.568270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:54.568270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:54.568270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:54.568270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:54.568270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:54.568270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:54.568270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:54.568270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:54.568270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:54.568270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:54.568270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:54.568270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:54.568270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:54.569394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:54.569394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:54.569394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:54.569394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:54.569867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 496/977] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:54.575812 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bdee22-2fbb-4b62-b6f7-f05b50df2b16 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:54.580126 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:54.653727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:54.670760 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de600f-f656-4006-b01e-8dbe83f61b99 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:54.687564 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bdee22-2fbb-4b62-b6f7-f05b50df2b16 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:54.709043 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700e6d6-276a-4457-944e-ff101cf952f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:54.709671 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700e6d6-276a-4457-944e-ff101cf952f1 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:54.709955 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700e6d6-276a-4457-944e-ff101cf952f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:54.710232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700e6d6-276a-4457-944e-ff101cf952f1 None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:54.710908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3700e6d6-276a-4457-944e-ff101cf952f1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:54.711213 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3700e6d6-276a-4457-944e-ff101cf952f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:54.713192 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3700e6d6-276a-4457-944e-ff101cf952f1 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:54.722979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 482/978] 10.222.0.39 () {56 vars in 1122 bytes} [Fri Aug 5 19:50:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:54.724424 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700e6d6-276a-4457-944e-ff101cf952f1 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:54.726661 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e01fb-e399-41d2-85b2-f53d73663908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:54.727282 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e01fb-e399-41d2-85b2-f53d73663908 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:54.727564 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e01fb-e399-41d2-85b2-f53d73663908 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:54.729257 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e01fb-e399-41d2-85b2-f53d73663908 None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:54.729257 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0e01fb-e399-41d2-85b2-f53d73663908 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:54.729564 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0e01fb-e399-41d2-85b2-f53d73663908 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:54.730930 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0e01fb-e399-41d2-85b2-f53d73663908 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:54.743068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 497/979] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:54.748856 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e01fb-e399-41d2-85b2-f53d73663908 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:54.817510 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:54.834894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700e6d6-276a-4457-944e-ff101cf952f1 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:54.864925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e01fb-e399-41d2-85b2-f53d73663908 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:54.869998 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d34e9-d34a-415e-8581-ad3289adeb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:54.870612 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d34e9-d34a-415e-8581-ad3289adeb99 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:54.870908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d34e9-d34a-415e-8581-ad3289adeb99 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:54.871183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d34e9-d34a-415e-8581-ad3289adeb99 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:54.871942 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d34e9-d34a-415e-8581-ad3289adeb99 None admin] RBAC: Adding query filter params (interface=admin, service_id=f771c5d061c24ba9b68d17cb90141708, region_id=RegionOne) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:54.872253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d34e9-d34a-415e-8581-ad3289adeb99 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:54.873648 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d34e9-d34a-415e-8581-ad3289adeb99 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:54.878799 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 483/980] 10.222.0.39 () {56 vars in 1249 bytes} [Fri Aug 5 19:50:54 2022] GET /identity/v3/endpoints?service_id=f771c5d061c24ba9b68d17cb90141708&interface=admin®ion_id=RegionOne => generated 193 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:54.884328 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d34e9-d34a-415e-8581-ad3289adeb99 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:54.902997 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:54.912066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eba868-2344-4849-9b56-9a0641ade768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:54.912739 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eba868-2344-4849-9b56-9a0641ade768 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:54.913011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eba868-2344-4849-9b56-9a0641ade768 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:54.913281 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eba868-2344-4849-9b56-9a0641ade768 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:54.920756 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eba868-2344-4849-9b56-9a0641ade768 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:54.922192 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eba868-2344-4849-9b56-9a0641ade768 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:54.935307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-24eba868-2344-4849-9b56-9a0641ade768 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:50:54.935307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:54.935307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:54.935307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:54.935307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:54.935307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:54.935307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:54.935307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:54.935307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:54.935307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:54.935307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:54.935307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:54.935307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:54.935307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:54.935307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:54.935307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:54.936356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:54.936356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:54.936356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:54.936356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:54.936356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:54.936356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:54.936356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:54.936356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:54.936356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:54.936356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:54.936356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:54.936356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:54.936356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:54.936356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:54.936356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:54.936356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:54.936356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:54.936356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:54.937996 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:54.937996 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:54.937996 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:54.937996 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:54.937996 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:54.937996 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:54.937996 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:54.937996 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:54.937996 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:50:54.937996 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:54.937996 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 498/981] 10.222.0.39 () {56 vars in 1117 bytes} [Fri Aug 5 19:50:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:54.942158 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eba868-2344-4849-9b56-9a0641ade768 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:55.012016 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d34e9-d34a-415e-8581-ad3289adeb99 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:55.050596 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c64bd-3b57-48bd-9580-7878e9cb98a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:55.051218 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c64bd-3b57-48bd-9580-7878e9cb98a8 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:55.051498 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c64bd-3b57-48bd-9580-7878e9cb98a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:55.051768 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c64bd-3b57-48bd-9580-7878e9cb98a8 None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:55.051867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eba868-2344-4849-9b56-9a0641ade768 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:55.052407 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c64bd-3b57-48bd-9580-7878e9cb98a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:55.054720 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c64bd-3b57-48bd-9580-7878e9cb98a8 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:55.073335 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 484/982] 10.222.0.39 () {56 vars in 1087 bytes} [Fri Aug 5 19:50:54 2022] GET /identity/v3/services => generated 1727 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:50:55.078662 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:55.092758 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23dfb9a-e2e1-456e-9662-94ec77a9ce7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:55.093408 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23dfb9a-e2e1-456e-9662-94ec77a9ce7a None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:55.093686 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23dfb9a-e2e1-456e-9662-94ec77a9ce7a None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:55.093970 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23dfb9a-e2e1-456e-9662-94ec77a9ce7a None admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:55.094783 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23dfb9a-e2e1-456e-9662-94ec77a9ce7a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:55.095090 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23dfb9a-e2e1-456e-9662-94ec77a9ce7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:55.096557 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23dfb9a-e2e1-456e-9662-94ec77a9ce7a None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:55.124482 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 499/983] 10.222.0.39 () {56 vars in 1149 bytes} [Fri Aug 5 19:50:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:55.125695 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c64bd-3b57-48bd-9580-7878e9cb98a8 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:55.217760 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:55.250759 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c64bd-3b57-48bd-9580-7878e9cb98a8 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:55.284756 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d5f39-18f3-4555-8508-0c91919b5d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:55.285379 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d5f39-18f3-4555-8508-0c91919b5d32 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:55.285660 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d5f39-18f3-4555-8508-0c91919b5d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:55.285936 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d5f39-18f3-4555-8508-0c91919b5d32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:55.354494 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d5f39-18f3-4555-8508-0c91919b5d32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:55.355472 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d5f39-18f3-4555-8508-0c91919b5d32 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:55.433476 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:55.461510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 485/984] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:50:55 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:50:55.467864 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23dfb9a-e2e1-456e-9662-94ec77a9ce7a None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:55.576085 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23dfb9a-e2e1-456e-9662-94ec77a9ce7a None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:55.613575 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d540cb96-fb33-4833-a21e-ad2cccc2135b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:55.614122 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d540cb96-fb33-4833-a21e-ad2cccc2135b None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:55.614402 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d540cb96-fb33-4833-a21e-ad2cccc2135b None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:55.614676 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d540cb96-fb33-4833-a21e-ad2cccc2135b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:55.625517 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d540cb96-fb33-4833-a21e-ad2cccc2135b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:55.626969 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d540cb96-fb33-4833-a21e-ad2cccc2135b None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:55.642414 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-d540cb96-fb33-4833-a21e-ad2cccc2135b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:55.642414 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:55.642414 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:55.642414 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:55.642414 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:55.642414 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:55.642414 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:55.642414 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:55.642414 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:55.642414 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:55.642414 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:55.642414 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:55.642414 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:55.642414 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:55.642414 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:55.643378 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:55.643378 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:55.643378 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:55.643378 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:55.643378 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:55.643378 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:55.643378 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:55.643378 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:55.643378 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:55.643378 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:55.643378 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:55.643378 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:55.643378 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:55.643378 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:55.643378 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:55.643378 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:55.643378 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:55.644289 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:55.644289 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:55.644289 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:55.644289 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:55.648624 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 500/985] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:55.649849 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d5f39-18f3-4555-8508-0c91919b5d32 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:55.755777 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d5f39-18f3-4555-8508-0c91919b5d32 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:55.793481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b857e1c-94bf-4ca8-9a10-f6648e2bfbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:55.794103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b857e1c-94bf-4ca8-9a10-f6648e2bfbe3 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:55.794430 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b857e1c-94bf-4ca8-9a10-f6648e2bfbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:55.794652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b857e1c-94bf-4ca8-9a10-f6648e2bfbe3 None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:55.795341 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b857e1c-94bf-4ca8-9a10-f6648e2bfbe3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:55.795645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b857e1c-94bf-4ca8-9a10-f6648e2bfbe3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:55.797106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b857e1c-94bf-4ca8-9a10-f6648e2bfbe3 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:55.810528 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 486/986] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:55.817647 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d540cb96-fb33-4833-a21e-ad2cccc2135b None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:55.855530 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:55.873782 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9fb592-66aa-4c69-a4dc-2cfbfb94b37e None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:55.873972 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9fb592-66aa-4c69-a4dc-2cfbfb94b37e None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:55.874249 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9fb592-66aa-4c69-a4dc-2cfbfb94b37e None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:55.875739 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 487/987] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:50:55.880441 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2742369-c551-4d47-a3fa-5a231df803d8 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:55.880810 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2742369-c551-4d47-a3fa-5a231df803d8 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:55.881096 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2742369-c551-4d47-a3fa-5a231df803d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:55.902147 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-d2742369-c551-4d47-a3fa-5a231df803d8 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:55.903400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-d2742369-c551-4d47-a3fa-5a231df803d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:55.932342 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d540cb96-fb33-4833-a21e-ad2cccc2135b None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:55.933031 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:55.972065 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99278f-772d-4ac8-b571-69100b850c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:55.972733 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99278f-772d-4ac8-b571-69100b850c7e None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:55.973019 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99278f-772d-4ac8-b571-69100b850c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:55.973294 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99278f-772d-4ac8-b571-69100b850c7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:55.979104 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 488/988] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:55 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:55.980320 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f99278f-772d-4ac8-b571-69100b850c7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:55.981798 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f99278f-772d-4ac8-b571-69100b850c7e None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:55.993314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-5f99278f-772d-4ac8-b571-69100b850c7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:55.993314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:55.993314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:55.993314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:55.993314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:55.993314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:55.993314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:55.993314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:55.993314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:55.993314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:55.993314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:55.993314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:55.993314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:55.993314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:55.993314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:55.994234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:55.994234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:55.994234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:55.994234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:55.994234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:55.994234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:55.994234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:55.994234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:55.994234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:55.994234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:55.994234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:55.994234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:55.994234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:55.994234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:55.994234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:55.994234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:55.994234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:55.995144 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:55.995144 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:55.995144 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:55.995144 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:55.995144 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:55.995144 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:55.995144 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:55.995144 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:55.996900 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 501/989] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:56.000841 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99278f-772d-4ac8-b571-69100b850c7e None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:56.000966 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2742369-c551-4d47-a3fa-5a231df803d8 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:56.100696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:56.108975 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99278f-772d-4ac8-b571-69100b850c7e None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:56.114786 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2742369-c551-4d47-a3fa-5a231df803d8 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:56.148466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a8664b-44c0-496d-ac79-1b39b277fb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:56.149082 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8664b-44c0-496d-ac79-1b39b277fb39 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:56.149364 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8664b-44c0-496d-ac79-1b39b277fb39 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:56.149642 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8664b-44c0-496d-ac79-1b39b277fb39 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:56.150864 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a8664b-44c0-496d-ac79-1b39b277fb39 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:56.152281 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a8664b-44c0-496d-ac79-1b39b277fb39 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:56.160611 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afb85c-3815-408d-9deb-99ef3bd25f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:56.161308 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afb85c-3815-408d-9deb-99ef3bd25f21 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:56.161593 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afb85c-3815-408d-9deb-99ef3bd25f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:56.161698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-e4a8664b-44c0-496d-ac79-1b39b277fb39 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 19:50:56.161698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:56.161698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:56.161698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:56.161698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:56.161698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:56.161698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:56.161698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:56.161698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 19:50:56.161698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:56.161698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:56.161698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:56.161698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:56.161698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:56.161698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:56.161698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:56.161698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:56.162675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:56.162675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:56.162675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:56.162675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:56.162675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:56.162675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:56.162675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:56.162675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:56.162675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:56.162675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:56.162675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:56.162675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:56.162675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:56.162675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:56.162675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:56.162675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:56.163868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:56.163868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:56.163868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:56.163868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:56.163868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:56.163868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:56.163868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:56.163868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:56.163868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:56.163868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:56.163868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:56.163868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:56.163868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 19:50:56.163868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:50:56.164658 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afb85c-3815-408d-9deb-99ef3bd25f21 None admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:56.164658 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3afb85c-3815-408d-9deb-99ef3bd25f21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:56.164787 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 502/990] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:56.164853 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3afb85c-3815-408d-9deb-99ef3bd25f21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:56.165487 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3afb85c-3815-408d-9deb-99ef3bd25f21 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:56.169720 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a8664b-44c0-496d-ac79-1b39b277fb39 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:56.178699 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 489/991] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:56.185600 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afb85c-3815-408d-9deb-99ef3bd25f21 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:56.221622 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:56.285396 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a8664b-44c0-496d-ac79-1b39b277fb39 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:56.311162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afb85c-3815-408d-9deb-99ef3bd25f21 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:56.325074 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0367be8-6e89-406b-a308-907c84db6cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:56.326082 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0367be8-6e89-406b-a308-907c84db6cfa None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:56.326565 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0367be8-6e89-406b-a308-907c84db6cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:56.327051 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0367be8-6e89-406b-a308-907c84db6cfa None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:56.327845 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0367be8-6e89-406b-a308-907c84db6cfa None admin] RBAC: Adding query filter params (name=compute) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:56.328247 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0367be8-6e89-406b-a308-907c84db6cfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:56.329750 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0367be8-6e89-406b-a308-907c84db6cfa None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:56.340615 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 503/992] 10.222.0.39 () {56 vars in 1112 bytes} [Fri Aug 5 19:50:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:56.341604 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0367be8-6e89-406b-a308-907c84db6cfa None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:56.350793 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0ec84-9913-443f-ba4d-949972b4ac08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:56.351493 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0ec84-9913-443f-ba4d-949972b4ac08 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:56.351843 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0ec84-9913-443f-ba4d-949972b4ac08 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:56.352182 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0ec84-9913-443f-ba4d-949972b4ac08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:56.354022 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd0ec84-9913-443f-ba4d-949972b4ac08 None admin] RBAC: Adding query filter params (role.id=0a40666ef2c140d1951dea50f3127507, scope.project.id=80379e2104644e98a2e85d4e4af1b365, user.id=3488a7d2258242c0859a105c1d162767) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:56.354410 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd0ec84-9913-443f-ba4d-949972b4ac08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:56.356215 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd0ec84-9913-443f-ba4d-949972b4ac08 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:56.372605 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 490/993] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:50:56 2022] GET /identity/v3/role_assignments?user.id=3488a7d2258242c0859a105c1d162767&scope.project.id=80379e2104644e98a2e85d4e4af1b365&role.id=0a40666ef2c140d1951dea50f3127507 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:56.376902 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:56.467683 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0367be8-6e89-406b-a308-907c84db6cfa None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:56.504782 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44753225-b8e6-4828-9d11-78ed0c131eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:56.505410 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44753225-b8e6-4828-9d11-78ed0c131eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:56.505690 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44753225-b8e6-4828-9d11-78ed0c131eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:56.505963 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44753225-b8e6-4828-9d11-78ed0c131eb9 None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:56.506642 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44753225-b8e6-4828-9d11-78ed0c131eb9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:56.506943 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44753225-b8e6-4828-9d11-78ed0c131eb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:56.508314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44753225-b8e6-4828-9d11-78ed0c131eb9 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:56.514699 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 504/994] 10.222.0.39 () {56 vars in 1112 bytes} [Fri Aug 5 19:50:56 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:56.520373 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0ec84-9913-443f-ba4d-949972b4ac08 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:56.623825 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:56.635406 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0ec84-9913-443f-ba4d-949972b4ac08 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:56.672270 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f838e-51b3-4998-91c2-725e89f3f9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:56.673028 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f838e-51b3-4998-91c2-725e89f3f9bf None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:56.673341 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f838e-51b3-4998-91c2-725e89f3f9bf None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:56.673685 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f838e-51b3-4998-91c2-725e89f3f9bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:56.674502 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651f838e-51b3-4998-91c2-725e89f3f9bf None admin] RBAC: Adding query filter params (interface=public, service_id=ad0e816bd72d488c8e94061ac5eac0e3, region_id=RegionOne) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:56.674882 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651f838e-51b3-4998-91c2-725e89f3f9bf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:56.676296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651f838e-51b3-4998-91c2-725e89f3f9bf None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:56.682358 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 491/995] 10.222.0.39 () {56 vars in 1251 bytes} [Fri Aug 5 19:50:56 2022] GET /identity/v3/endpoints?service_id=ad0e816bd72d488c8e94061ac5eac0e3&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:56.687893 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44753225-b8e6-4828-9d11-78ed0c131eb9 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:56.779608 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:56.796366 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44753225-b8e6-4828-9d11-78ed0c131eb9 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:56.831426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca730381-c187-49c8-8c3f-a4b55d29a27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:56.832268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca730381-c187-49c8-8c3f-a4b55d29a27c None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:56.832639 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca730381-c187-49c8-8c3f-a4b55d29a27c None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:56.832980 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca730381-c187-49c8-8c3f-a4b55d29a27c None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:56.833697 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca730381-c187-49c8-8c3f-a4b55d29a27c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:56.835158 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca730381-c187-49c8-8c3f-a4b55d29a27c None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:56.848693 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 505/996] 10.222.0.39 () {56 vars in 1087 bytes} [Fri Aug 5 19:50:56 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:57.216372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db6a10-571e-4e79-aeec-05e597394291 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:57.216841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db6a10-571e-4e79-aeec-05e597394291 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:57.217219 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db6a10-571e-4e79-aeec-05e597394291 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:57.219008 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 492/997] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:57.224115 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6fb45-dfa9-4fdd-b04c-c903a5ca7a24 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:57.224509 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6fb45-dfa9-4fdd-b04c-c903a5ca7a24 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:57.224913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6fb45-dfa9-4fdd-b04c-c903a5ca7a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:57.246539 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-48f6fb45-dfa9-4fdd-b04c-c903a5ca7a24 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:57.247720 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-48f6fb45-dfa9-4fdd-b04c-c903a5ca7a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:57.319147 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 506/998] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:57 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:57.345767 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3db6a10-571e-4e79-aeec-05e597394291 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:57.473455 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3db6a10-571e-4e79-aeec-05e597394291 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:57.518507 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bc930c-f702-4e81-990c-5e855a36ffa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:57.519503 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc930c-f702-4e81-990c-5e855a36ffa5 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:57.519783 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc930c-f702-4e81-990c-5e855a36ffa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:57.520058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc930c-f702-4e81-990c-5e855a36ffa5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:57.520724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bc930c-f702-4e81-990c-5e855a36ffa5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:57.522083 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bc930c-f702-4e81-990c-5e855a36ffa5 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:57.534567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-e2bc930c-f702-4e81-990c-5e855a36ffa5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:50:57.534567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:57.534567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:57.534567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:57.534567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:57.534567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:57.534567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:57.534567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:57.534567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:50:57.534567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:57.534567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:57.534567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:57.534567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:57.534567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:57.534567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:57.534567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:57.534567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:57.535766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:57.535766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:57.535766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:57.535766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:57.535766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:57.535766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:57.535766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:57.535766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:57.535766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:57.535766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:57.535766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:57.535766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:57.535766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:57.535766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:57.535766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:57.535766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:57.542309 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:57.542309 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:57.542309 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:57.542309 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:57.542309 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:57.542309 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:57.542309 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:57.542309 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:57.542309 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:57.542309 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:57.542309 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:57.542309 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:57.542309 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:50:57.542309 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:57.542309 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 493/999] 10.222.0.39 () {56 vars in 1126 bytes} [Fri Aug 5 19:50:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:57.546401 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6fb45-dfa9-4fdd-b04c-c903a5ca7a24 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:57.640983 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:57.664451 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6fb45-dfa9-4fdd-b04c-c903a5ca7a24 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:57.706100 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec5237-541e-483f-9090-ea38b940eea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:57.706721 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec5237-541e-483f-9090-ea38b940eea5 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:57.706997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec5237-541e-483f-9090-ea38b940eea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:57.707268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec5237-541e-483f-9090-ea38b940eea5 None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:57.707944 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec5237-541e-483f-9090-ea38b940eea5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:57.708248 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec5237-541e-483f-9090-ea38b940eea5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:57.709648 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec5237-541e-483f-9090-ea38b940eea5 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:57.720611 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 507/1000] 10.222.0.39 () {56 vars in 1122 bytes} [Fri Aug 5 19:50:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:57.722283 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bc930c-f702-4e81-990c-5e855a36ffa5 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:57.753007 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:57.854734 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bc930c-f702-4e81-990c-5e855a36ffa5 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:57.893762 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc2b19-e75d-420e-ac48-3e5a0653a057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:57.894801 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc2b19-e75d-420e-ac48-3e5a0653a057 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:57.895338 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc2b19-e75d-420e-ac48-3e5a0653a057 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:57.895618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc2b19-e75d-420e-ac48-3e5a0653a057 None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:57.896297 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bc2b19-e75d-420e-ac48-3e5a0653a057 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:57.896626 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bc2b19-e75d-420e-ac48-3e5a0653a057 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:57.898254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bc2b19-e75d-420e-ac48-3e5a0653a057 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:57.908974 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 494/1001] 10.222.0.39 () {56 vars in 1122 bytes} [Fri Aug 5 19:50:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:57.910456 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec5237-541e-483f-9090-ea38b940eea5 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:58.068370 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec5237-541e-483f-9090-ea38b940eea5 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:58.128414 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287610f-5009-4b06-8faf-53cd3f4ead52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:58.129125 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287610f-5009-4b06-8faf-53cd3f4ead52 None admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:58.129369 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287610f-5009-4b06-8faf-53cd3f4ead52 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:58.129861 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287610f-5009-4b06-8faf-53cd3f4ead52 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:58.130488 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1287610f-5009-4b06-8faf-53cd3f4ead52 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:58.131893 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1287610f-5009-4b06-8faf-53cd3f4ead52 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:58.164676 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 508/1002] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:57 2022] POST /identity/v3/endpoints => generated 325 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:58.504679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f1087-2e55-490b-9285-a8eadc7be956 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:58.504973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f1087-2e55-490b-9285-a8eadc7be956 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:58.505307 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f1087-2e55-490b-9285-a8eadc7be956 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:58.507004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 495/1003] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:50:58.513185 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ef76c-3fa5-45da-8f87-31f623ef93d2 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:58.513465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ef76c-3fa5-45da-8f87-31f623ef93d2 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:58.513735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ef76c-3fa5-45da-8f87-31f623ef93d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:58.521468 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:58.539367 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-f82ef76c-3fa5-45da-8f87-31f623ef93d2 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:58.542354 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-f82ef76c-3fa5-45da-8f87-31f623ef93d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:58.695649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 509/1004] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:58 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:58.732015 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f1087-2e55-490b-9285-a8eadc7be956 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:58.876444 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:58.902863 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f1087-2e55-490b-9285-a8eadc7be956 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:58.953757 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732af6e3-9ae9-41b5-a50b-6bae9b448ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:58.955714 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732af6e3-9ae9-41b5-a50b-6bae9b448ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:58.955714 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732af6e3-9ae9-41b5-a50b-6bae9b448ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:58.955714 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732af6e3-9ae9-41b5-a50b-6bae9b448ce3 None admin] PATH_INFO: `/v3/services/image` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:58.955714 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732af6e3-9ae9-41b5-a50b-6bae9b448ce3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:58.957100 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732af6e3-9ae9-41b5-a50b-6bae9b448ce3 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:58.972270 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-732af6e3-9ae9-41b5-a50b-6bae9b448ce3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 19:50:58.972270 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:58.972270 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:58.972270 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:58.972270 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:58.972270 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:58.972270 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:58.972270 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:58.972270 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 19:50:58.972270 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:58.972270 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:58.972270 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:58.972270 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:58.972270 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:58.972270 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:58.972270 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:58.972270 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:58.974065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:58.974065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:58.974065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:58.974065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:58.974065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:58.974065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:58.974065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:58.974065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:58.974065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:58.974065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:58.974065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:58.974065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:58.974065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:58.974065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:58.974065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:58.974065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:58.975724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:58.975724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:58.975724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:58.975724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:58.975724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:58.975724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:58.975724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:58.975724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:58.975724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:58.975724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:58.975724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:58.975724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:58.975724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 19:50:58.975724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:58.980694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 496/1005] 10.222.0.39 () {56 vars in 1105 bytes} [Fri Aug 5 19:50:58 2022] GET /identity/v3/services/image => generated 86 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:58.984931 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ef76c-3fa5-45da-8f87-31f623ef93d2 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:59.133148 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ef76c-3fa5-45da-8f87-31f623ef93d2 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:59.141960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3df06-1185-4104-b913-a6186523590f None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:59.141960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3df06-1185-4104-b913-a6186523590f None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:59.141960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3df06-1185-4104-b913-a6186523590f None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:59.141960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 497/1006] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:50:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:50:59.148531 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0468ae6-c875-437f-a81e-2e22db5d778a None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:59.148865 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0468ae6-c875-437f-a81e-2e22db5d778a None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:59.149145 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0468ae6-c875-437f-a81e-2e22db5d778a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:59.168819 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-e0468ae6-c875-437f-a81e-2e22db5d778a None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:59.170024 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-e0468ae6-c875-437f-a81e-2e22db5d778a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:59.188381 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:59.196234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475f58b-3d1c-4347-99c4-5fe6bac8b227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:59.197217 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475f58b-3d1c-4347-99c4-5fe6bac8b227 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:59.197709 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475f58b-3d1c-4347-99c4-5fe6bac8b227 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:59.198290 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475f58b-3d1c-4347-99c4-5fe6bac8b227 None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:59.199550 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4475f58b-3d1c-4347-99c4-5fe6bac8b227 None admin] RBAC: Adding query filter params (name=image) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:59.200063 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4475f58b-3d1c-4347-99c4-5fe6bac8b227 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:59.202208 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4475f58b-3d1c-4347-99c4-5fe6bac8b227 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:59.208244 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 510/1007] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:59.214636 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475f58b-3d1c-4347-99c4-5fe6bac8b227 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:59.266458 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 498/1008] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:59 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:50:59.289802 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0468ae6-c875-437f-a81e-2e22db5d778a None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:59.344155 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475f58b-3d1c-4347-99c4-5fe6bac8b227 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:59.382908 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59766e8-4328-4d3b-95f3-aa24a69894fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:59.383609 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59766e8-4328-4d3b-95f3-aa24a69894fe None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:59.383825 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59766e8-4328-4d3b-95f3-aa24a69894fe None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:59.384117 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59766e8-4328-4d3b-95f3-aa24a69894fe None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:59.385072 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:59.385306 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59766e8-4328-4d3b-95f3-aa24a69894fe None admin] RBAC: Adding query filter params (type=image) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:59.385602 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59766e8-4328-4d3b-95f3-aa24a69894fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:59.387839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59766e8-4328-4d3b-95f3-aa24a69894fe None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:59.392999 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 511/1009] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:59 2022] GET /identity/v3/services?type=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:59.424267 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0468ae6-c875-437f-a81e-2e22db5d778a None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:59.475259 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a472b4b-3c25-4239-9007-d0386385245a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:59.476073 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a472b4b-3c25-4239-9007-d0386385245a None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:59.476360 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a472b4b-3c25-4239-9007-d0386385245a None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:59.476680 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a472b4b-3c25-4239-9007-d0386385245a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:59.477373 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a472b4b-3c25-4239-9007-d0386385245a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:59.480286 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a472b4b-3c25-4239-9007-d0386385245a None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:59.491004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-5a472b4b-3c25-4239-9007-d0386385245a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 19:50:59.491004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:59.491004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:59.491004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:59.491004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:59.491004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:59.491004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:59.491004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:59.491004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 19:50:59.491004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:59.491004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:59.491004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:59.491004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:59.491004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:59.491004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:59.491004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:59.491004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:59.492112 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:59.492112 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:59.492112 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:59.492112 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:59.492112 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:59.492112 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:59.492112 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:59.492112 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:59.492112 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:59.492112 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:59.492112 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:59.492112 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:59.492112 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:59.492112 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:59.492112 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:59.492112 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:59.493343 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:59.493343 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:59.493343 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:59.493343 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:59.493343 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:59.493343 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:59.493343 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:59.493343 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:59.493343 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:59.493343 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:59.493343 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:59.493343 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:59.493343 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 19:50:59.493343 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:50:59.493343 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 499/1010] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:59.500855 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59766e8-4328-4d3b-95f3-aa24a69894fe None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:59.578192 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:59.631426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59766e8-4328-4d3b-95f3-aa24a69894fe None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:59.674513 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9646a5-5cee-45ec-9b1e-8d5507804b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:59.675036 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9646a5-5cee-45ec-9b1e-8d5507804b46 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:59.675311 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9646a5-5cee-45ec-9b1e-8d5507804b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:59.675578 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9646a5-5cee-45ec-9b1e-8d5507804b46 None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:59.676262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9646a5-5cee-45ec-9b1e-8d5507804b46 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:59.676554 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9646a5-5cee-45ec-9b1e-8d5507804b46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:59.678171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9646a5-5cee-45ec-9b1e-8d5507804b46 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:59.688377 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 512/1011] 10.222.0.39 () {56 vars in 1112 bytes} [Fri Aug 5 19:50:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:59.689825 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a472b4b-3c25-4239-9007-d0386385245a None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:59.805642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a472b4b-3c25-4239-9007-d0386385245a None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:59.844913 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5dadec-bbd1-40d2-86d3-26a7d5dd69ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:59.845529 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5dadec-bbd1-40d2-86d3-26a7d5dd69ef None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:59.845806 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5dadec-bbd1-40d2-86d3-26a7d5dd69ef None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:59.846075 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5dadec-bbd1-40d2-86d3-26a7d5dd69ef None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:59.846750 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5dadec-bbd1-40d2-86d3-26a7d5dd69ef None admin] RBAC: Adding query filter params (type=compute) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:59.847049 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5dadec-bbd1-40d2-86d3-26a7d5dd69ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:59.848424 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5dadec-bbd1-40d2-86d3-26a7d5dd69ef None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:59.854143 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 500/1012] 10.222.0.39 () {56 vars in 1112 bytes} [Fri Aug 5 19:50:59 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:59.859947 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9646a5-5cee-45ec-9b1e-8d5507804b46 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:59.976393 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9646a5-5cee-45ec-9b1e-8d5507804b46 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:00.004018 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:00.018849 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e552a4f9-9921-4ecd-8b80-603d06db9358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:00.019634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552a4f9-9921-4ecd-8b80-603d06db9358 None admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:00.019978 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552a4f9-9921-4ecd-8b80-603d06db9358 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:00.020319 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552a4f9-9921-4ecd-8b80-603d06db9358 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:00.021155 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e552a4f9-9921-4ecd-8b80-603d06db9358 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:00.022834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e552a4f9-9921-4ecd-8b80-603d06db9358 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:00.051275 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 513/1013] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:59 2022] POST /identity/v3/endpoints => generated 334 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:51:00.432755 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33357b6f-3c50-4906-b8f2-2fc6a25930d5 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:00.433065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33357b6f-3c50-4906-b8f2-2fc6a25930d5 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:00.433318 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33357b6f-3c50-4906-b8f2-2fc6a25930d5 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:00.434707 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 501/1014] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:51:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:00.440828 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7acd7-0c24-436b-b848-bc4d07a456dc None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:00.441187 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7acd7-0c24-436b-b848-bc4d07a456dc None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:00.441540 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7acd7-0c24-436b-b848-bc4d07a456dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:00.462247 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-77c7acd7-0c24-436b-b848-bc4d07a456dc None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:00.464123 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-77c7acd7-0c24-436b-b848-bc4d07a456dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:00.557277 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 514/1015] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:00 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:51:00.581948 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33357b6f-3c50-4906-b8f2-2fc6a25930d5 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:00.587295 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:00.725432 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33357b6f-3c50-4906-b8f2-2fc6a25930d5 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:00.778863 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbfb3a-2a84-46ae-9e5d-0b122f61aae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:00.779529 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbfb3a-2a84-46ae-9e5d-0b122f61aae5 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:00.779811 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbfb3a-2a84-46ae-9e5d-0b122f61aae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:00.780087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbfb3a-2a84-46ae-9e5d-0b122f61aae5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:00.786565 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbfb3a-2a84-46ae-9e5d-0b122f61aae5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:00.788033 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbfb3a-2a84-46ae-9e5d-0b122f61aae5 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:00.789721 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 502/1016] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:51:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:00.796177 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c7acd7-0c24-436b-b848-bc4d07a456dc None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:00.911423 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c7acd7-0c24-436b-b848-bc4d07a456dc None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:00.913963 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:00.955911 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3bf83-59d3-4f83-ba5f-34c33b3864b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:00.956548 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3bf83-59d3-4f83-ba5f-34c33b3864b6 None admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:00.957039 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3bf83-59d3-4f83-ba5f-34c33b3864b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:00.957315 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3bf83-59d3-4f83-ba5f-34c33b3864b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:00.958019 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3bf83-59d3-4f83-ba5f-34c33b3864b6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:00.959476 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3bf83-59d3-4f83-ba5f-34c33b3864b6 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:01.002233 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 515/1017] 10.222.0.39 () {60 vars in 1141 bytes} [Fri Aug 5 19:51:00 2022] POST /identity/v3/projects => generated 311 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:51:01.636178 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d80db0-ef0e-44aa-9427-2b85ab96f078 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:01.636836 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d80db0-ef0e-44aa-9427-2b85ab96f078 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:01.636836 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d80db0-ef0e-44aa-9427-2b85ab96f078 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:01.638273 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 503/1018] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:51:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:01.643473 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f28198-754e-4dba-84de-826ff7c37726 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:01.643909 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f28198-754e-4dba-84de-826ff7c37726 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:01.644079 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f28198-754e-4dba-84de-826ff7c37726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:01.662982 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-a0f28198-754e-4dba-84de-826ff7c37726 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:01.664154 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-a0f28198-754e-4dba-84de-826ff7c37726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:01.739153 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 516/1019] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:01 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:51:01.763004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d80db0-ef0e-44aa-9427-2b85ab96f078 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:01.840489 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:01.878798 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d80db0-ef0e-44aa-9427-2b85ab96f078 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:01.914604 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2e90f9-2c76-40ac-9efc-1420c74adba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:01.915136 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e90f9-2c76-40ac-9efc-1420c74adba0 None admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:01.915592 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e90f9-2c76-40ac-9efc-1420c74adba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:01.916411 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e90f9-2c76-40ac-9efc-1420c74adba0 None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:01.917144 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2e90f9-2c76-40ac-9efc-1420c74adba0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:01.918498 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2e90f9-2c76-40ac-9efc-1420c74adba0 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:01.934413 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 504/1020] 10.222.0.39 () {60 vars in 1141 bytes} [Fri Aug 5 19:51:01 2022] POST /identity/v3/services => generated 242 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:51:02.343148 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171bdebd-d964-43f9-b5de-e0f357aa134d None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:02.343454 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171bdebd-d964-43f9-b5de-e0f357aa134d None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:02.343620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171bdebd-d964-43f9-b5de-e0f357aa134d None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:02.345073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 517/1021] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:02.350277 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9cbe0-1bae-486f-9d9a-f0043e1c0304 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:02.350574 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9cbe0-1bae-486f-9d9a-f0043e1c0304 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:02.350868 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9cbe0-1bae-486f-9d9a-f0043e1c0304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:02.370390 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-f7d9cbe0-1bae-486f-9d9a-f0043e1c0304 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:02.371594 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-f7d9cbe0-1bae-486f-9d9a-f0043e1c0304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:02.453494 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 505/1022] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:02 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:51:02.476429 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171bdebd-d964-43f9-b5de-e0f357aa134d None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:02.569467 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:02.589601 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171bdebd-d964-43f9-b5de-e0f357aa134d None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:02.629338 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8307f66c-d7be-4c74-b71f-30422a7d09b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:02.629550 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307f66c-d7be-4c74-b71f-30422a7d09b7 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:02.629804 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307f66c-d7be-4c74-b71f-30422a7d09b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:02.629978 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307f66c-d7be-4c74-b71f-30422a7d09b7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:02.641906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8307f66c-d7be-4c74-b71f-30422a7d09b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:51:02.646655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-8307f66c-d7be-4c74-b71f-30422a7d09b7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:51:02.646655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:02.646655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:02.646655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:02.646655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:02.646655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:02.646655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:02.646655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:02.646655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:02.646655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:02.646655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:02.646655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:02.646655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:51:02.646655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:51:02.646655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:51:02.646655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:51:02.648301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:51:02.648301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:51:02.648301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:51:02.648301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:51:02.648301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:02.648301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:02.648301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:02.648301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:02.648301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:02.648301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:02.648301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:02.648301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:02.648301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:02.648301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:02.648301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:02.648301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:02.649207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:02.649207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:02.649207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:02.649207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:02.649207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:51:02.649207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:51:02.649207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:51:02.649207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:51:02.649207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:51:02.649207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:51:02.649207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:51:02.649207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:51:02.652668 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 518/1023] 10.222.0.39 () {56 vars in 1120 bytes} [Fri Aug 5 19:51:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:51:02.655115 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9cbe0-1bae-486f-9d9a-f0043e1c0304 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:02.721080 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:02.768356 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9cbe0-1bae-486f-9d9a-f0043e1c0304 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:02.804385 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de817974-5d62-4863-941b-4efaf61c5abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:02.805038 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de817974-5d62-4863-941b-4efaf61c5abf None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:02.805320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de817974-5d62-4863-941b-4efaf61c5abf None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:02.805595 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de817974-5d62-4863-941b-4efaf61c5abf None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:02.807053 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de817974-5d62-4863-941b-4efaf61c5abf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:02.807359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de817974-5d62-4863-941b-4efaf61c5abf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:02.808753 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de817974-5d62-4863-941b-4efaf61c5abf None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:02.817495 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 506/1024] 10.222.0.39 () {56 vars in 1115 bytes} [Fri Aug 5 19:51:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:02.823500 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8307f66c-d7be-4c74-b71f-30422a7d09b7 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:02.929351 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8307f66c-d7be-4c74-b71f-30422a7d09b7 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:02.964393 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256fa36-05aa-44af-b6e9-d57d0bbca07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:02.965020 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256fa36-05aa-44af-b6e9-d57d0bbca07f None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:02.965303 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256fa36-05aa-44af-b6e9-d57d0bbca07f None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:02.965578 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256fa36-05aa-44af-b6e9-d57d0bbca07f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:02.976745 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e256fa36-05aa-44af-b6e9-d57d0bbca07f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:02.978782 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e256fa36-05aa-44af-b6e9-d57d0bbca07f None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:02.985429 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:02.995682 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-e256fa36-05aa-44af-b6e9-d57d0bbca07f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:51:02.995682 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:02.995682 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:02.995682 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:02.995682 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:02.995682 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:02.995682 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:02.995682 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:02.995682 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:02.995682 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:02.995682 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:02.995682 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:02.995682 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:51:02.995682 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:51:02.995682 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:51:02.996686 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:51:02.996686 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:02.996686 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:02.996686 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:02.996686 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:02.996686 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:02.996686 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:02.996686 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:02.996686 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:02.996686 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:02.996686 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:02.996686 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:02.996686 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:02.996686 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:02.996686 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:02.996686 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:02.996686 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:02.997647 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:51:02.997647 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:51:02.997647 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:51:02.997647 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:51:02.997647 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 519/1025] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:51:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:51:03.003965 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de817974-5d62-4863-941b-4efaf61c5abf None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:03.121169 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de817974-5d62-4863-941b-4efaf61c5abf None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:03.138753 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02f497-e0b4-45e0-83e2-5c6f6cd57a5e None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:03.139493 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02f497-e0b4-45e0-83e2-5c6f6cd57a5e None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:03.139773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02f497-e0b4-45e0-83e2-5c6f6cd57a5e None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:03.141260 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 520/1026] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:51:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:03.149032 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2977b-ada6-45c4-9321-a729d9e8be17 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:03.149366 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2977b-ada6-45c4-9321-a729d9e8be17 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:03.149660 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2977b-ada6-45c4-9321-a729d9e8be17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:03.151554 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:03.167921 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b088c4-dcfb-45e4-b707-c57913649830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:03.168538 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b088c4-dcfb-45e4-b707-c57913649830 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:03.168848 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b088c4-dcfb-45e4-b707-c57913649830 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:03.169122 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b088c4-dcfb-45e4-b707-c57913649830 None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:03.170840 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-72a2977b-ada6-45c4-9321-a729d9e8be17 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:03.171363 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b088c4-dcfb-45e4-b707-c57913649830 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:03.171691 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b088c4-dcfb-45e4-b707-c57913649830 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:03.172076 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-72a2977b-ada6-45c4-9321-a729d9e8be17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:03.173083 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b088c4-dcfb-45e4-b707-c57913649830 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:03.187901 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 507/1027] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:51:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:03.193311 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b088c4-dcfb-45e4-b707-c57913649830 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:03.256613 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 521/1028] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:03 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:51:03.278008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2977b-ada6-45c4-9321-a729d9e8be17 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:03.312439 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b088c4-dcfb-45e4-b707-c57913649830 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:03.353570 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f3a3f-2365-48e0-8de7-1685ff5ec785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:03.354560 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f3a3f-2365-48e0-8de7-1685ff5ec785 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:03.354845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f3a3f-2365-48e0-8de7-1685ff5ec785 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:03.355124 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f3a3f-2365-48e0-8de7-1685ff5ec785 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:03.363235 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f3a3f-2365-48e0-8de7-1685ff5ec785 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:03.364757 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f3a3f-2365-48e0-8de7-1685ff5ec785 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:03.378368 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-772f3a3f-2365-48e0-8de7-1685ff5ec785 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:51:03.378368 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:03.378368 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:03.378368 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:03.378368 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:03.378368 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:03.378368 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:03.378368 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:03.378368 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:03.378368 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:03.378368 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:03.378368 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:03.378368 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:51:03.378368 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:51:03.378368 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:51:03.378368 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:51:03.379773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:03.379773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:03.379773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:51:03.379773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:51:03.379773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:51:03.379773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:51:03.379773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:03.379773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:03.379773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:03.379773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:03.379773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:03.379773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:03.379773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:03.379773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:03.379773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:03.379773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:03.379773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:03.379773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:03.380861 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:03.380861 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:03.380861 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:51:03.380861 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:51:03.380861 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:51:03.380861 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:51:03.380861 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:51:03.380861 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:51:03.380861 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:51:03.380861 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:51:03.380861 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 508/1029] 10.222.0.39 () {56 vars in 1093 bytes} [Fri Aug 5 19:51:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:51:03.386059 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f3a3f-2365-48e0-8de7-1685ff5ec785 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:03.389239 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2977b-ada6-45c4-9321-a729d9e8be17 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:03.422390 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:03.431693 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eedc50c-286f-4dd1-92a7-14cc0055992c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:03.432440 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedc50c-286f-4dd1-92a7-14cc0055992c None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:03.432768 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedc50c-286f-4dd1-92a7-14cc0055992c None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:03.433037 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedc50c-286f-4dd1-92a7-14cc0055992c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:03.434185 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eedc50c-286f-4dd1-92a7-14cc0055992c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:03.435661 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eedc50c-286f-4dd1-92a7-14cc0055992c None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:03.438337 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 522/1030] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:51:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:51:03.443130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:03.443618 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eedc50c-286f-4dd1-92a7-14cc0055992c None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:03.516051 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f3a3f-2365-48e0-8de7-1685ff5ec785 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:03.560494 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eedc50c-286f-4dd1-92a7-14cc0055992c None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:03.562478 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ccc193-b70a-48bf-9dc0-795cfb01ac78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:03.563104 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ccc193-b70a-48bf-9dc0-795cfb01ac78 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:03.563387 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ccc193-b70a-48bf-9dc0-795cfb01ac78 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:03.563663 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ccc193-b70a-48bf-9dc0-795cfb01ac78 None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:03.564467 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ccc193-b70a-48bf-9dc0-795cfb01ac78 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:03.564799 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ccc193-b70a-48bf-9dc0-795cfb01ac78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:03.566237 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ccc193-b70a-48bf-9dc0-795cfb01ac78 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:03.592047 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 509/1031] 10.222.0.39 () {56 vars in 1133 bytes} [Fri Aug 5 19:51:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:03.593329 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ccc193-b70a-48bf-9dc0-795cfb01ac78 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:03.599213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cda238-2d49-49f3-abfa-e21691d31b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:03.599957 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cda238-2d49-49f3-abfa-e21691d31b29 None admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:03.600141 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cda238-2d49-49f3-abfa-e21691d31b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:03.600421 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cda238-2d49-49f3-abfa-e21691d31b29 None admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:03.601083 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cda238-2d49-49f3-abfa-e21691d31b29 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:03.603172 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cda238-2d49-49f3-abfa-e21691d31b29 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:03.636613 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 523/1032] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Aug 5 19:51:03 2022] POST /identity/v3/users => generated 285 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:51:03.724915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:03.731325 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ccc193-b70a-48bf-9dc0-795cfb01ac78 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:03.770907 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34bfbe-45ca-4fb4-af1e-d28ac33e486c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:03.771536 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34bfbe-45ca-4fb4-af1e-d28ac33e486c None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:03.771824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34bfbe-45ca-4fb4-af1e-d28ac33e486c None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:03.772100 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34bfbe-45ca-4fb4-af1e-d28ac33e486c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:03.849726 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34bfbe-45ca-4fb4-af1e-d28ac33e486c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:03.851192 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34bfbe-45ca-4fb4-af1e-d28ac33e486c None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:03.960066 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:03.965960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 510/1033] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:51:03 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:51:03.972692 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cda238-2d49-49f3-abfa-e21691d31b29 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:04.032476 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:04.086351 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cda238-2d49-49f3-abfa-e21691d31b29 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:04.104335 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e5a1d-23ba-4fdb-a795-e7c4b02a6da3 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:04.104731 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e5a1d-23ba-4fdb-a795-e7c4b02a6da3 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:04.105080 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e5a1d-23ba-4fdb-a795-e7c4b02a6da3 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:04.106709 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 511/1034] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:51:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:04.111718 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9d795-88b2-4243-bda0-3148b23f16f8 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:04.114902 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9d795-88b2-4243-bda0-3148b23f16f8 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:04.115176 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9d795-88b2-4243-bda0-3148b23f16f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:04.127651 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8dbdc4-f149-492d-bfe8-f03d5648b18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:04.128296 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8dbdc4-f149-492d-bfe8-f03d5648b18e None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:04.128616 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8dbdc4-f149-492d-bfe8-f03d5648b18e None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:04.128885 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8dbdc4-f149-492d-bfe8-f03d5648b18e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:04.138861 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-08b9d795-88b2-4243-bda0-3148b23f16f8 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:04.142224 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8dbdc4-f149-492d-bfe8-f03d5648b18e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:04.143474 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-08b9d795-88b2-4243-bda0-3148b23f16f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:04.143696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8dbdc4-f149-492d-bfe8-f03d5648b18e None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:04.160072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-9c8dbdc4-f149-492d-bfe8-f03d5648b18e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:51:04.160072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:04.160072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:04.160072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:04.160072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:04.160072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:04.160072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:04.160072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:04.160072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:04.160072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:04.160072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:04.160072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:04.160072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:51:04.160072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:51:04.160072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:51:04.161120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:51:04.161120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:04.161120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:04.161120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:04.161120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:04.161120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:04.161120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:04.161120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:04.161120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:04.161120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:04.161120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:04.161120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:04.161120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:04.161120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:04.161120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:04.161120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:04.161120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:04.162116 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:51:04.162116 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:51:04.162116 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:51:04.162116 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:51:04.162116 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 524/1035] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:51:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:51:04.167296 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8dbdc4-f149-492d-bfe8-f03d5648b18e None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:04.225359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 512/1036] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:51:04.249044 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b9d795-88b2-4243-bda0-3148b23f16f8 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:04.286899 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8dbdc4-f149-492d-bfe8-f03d5648b18e None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:04.291073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:04.329222 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e50dea-f9b5-45a0-8f27-c94332fdad44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:04.329854 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e50dea-f9b5-45a0-8f27-c94332fdad44 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:04.330120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e50dea-f9b5-45a0-8f27-c94332fdad44 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:04.330398 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e50dea-f9b5-45a0-8f27-c94332fdad44 None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:04.331099 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e50dea-f9b5-45a0-8f27-c94332fdad44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:04.331405 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e50dea-f9b5-45a0-8f27-c94332fdad44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:04.332841 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e50dea-f9b5-45a0-8f27-c94332fdad44 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:04.350075 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 525/1037] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:04.351182 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e50dea-f9b5-45a0-8f27-c94332fdad44 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:04.378026 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b9d795-88b2-4243-bda0-3148b23f16f8 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:04.411490 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:04.420470 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f3ae8c-e1ec-463b-9787-7de5d821c143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:04.421167 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3ae8c-e1ec-463b-9787-7de5d821c143 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:04.421452 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3ae8c-e1ec-463b-9787-7de5d821c143 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:04.421730 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3ae8c-e1ec-463b-9787-7de5d821c143 None admin] PATH_INFO: `/v3/services/image` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:04.422369 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f3ae8c-e1ec-463b-9787-7de5d821c143 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:04.423756 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f3ae8c-e1ec-463b-9787-7de5d821c143 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:04.433719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-f0f3ae8c-e1ec-463b-9787-7de5d821c143 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 19:51:04.433719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:04.433719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:51:04.433719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:51:04.433719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:51:04.433719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:51:04.433719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:51:04.433719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:51:04.433719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 19:51:04.433719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:51:04.433719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:51:04.433719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:51:04.433719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:04.433719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:04.433719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:04.433719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:04.433719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:04.434771 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:04.434771 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:04.434771 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:04.434771 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:04.434771 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:04.434771 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:04.434771 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:51:04.434771 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:51:04.434771 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:51:04.434771 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:51:04.434771 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:04.434771 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:04.434771 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:04.434771 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:04.434771 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:04.434771 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:04.435643 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:04.435643 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:04.435643 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:04.435643 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:04.435643 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:04.435643 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:04.435643 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:04.435643 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:04.435643 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:04.435643 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:04.435643 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:51:04.435643 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:51:04.435643 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 19:51:04.435643 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:51:04.435643 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 513/1038] 10.222.0.39 () {56 vars in 1105 bytes} [Fri Aug 5 19:51:04 2022] GET /identity/v3/services/image => generated 86 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:51:04.440856 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f3ae8c-e1ec-463b-9787-7de5d821c143 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:04.474263 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e50dea-f9b5-45a0-8f27-c94332fdad44 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:04.513243 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e6bc3f-57df-4216-a282-2f51df5a3a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:04.513895 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6bc3f-57df-4216-a282-2f51df5a3a2d None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:04.514186 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6bc3f-57df-4216-a282-2f51df5a3a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:04.514470 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6bc3f-57df-4216-a282-2f51df5a3a2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:04.521853 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e6bc3f-57df-4216-a282-2f51df5a3a2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:04.523585 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e6bc3f-57df-4216-a282-2f51df5a3a2d None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:04.536219 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-94e6bc3f-57df-4216-a282-2f51df5a3a2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:51:04.536219 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:04.536219 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:04.536219 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:04.536219 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:04.536219 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:04.536219 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:04.536219 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:04.536219 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:04.536219 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:04.536219 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:04.536219 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:04.536219 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:51:04.536219 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:51:04.536219 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:51:04.537466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:51:04.537466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:04.537466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:04.537466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:04.537466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:04.537466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:04.537466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:04.537466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:04.537466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:04.537466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:04.537466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:04.537466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:04.537466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:04.537466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:04.537466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:04.537466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:04.537466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:04.544255 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:51:04.544255 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:51:04.544255 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:51:04.544255 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:51:04.544255 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:51:04.544255 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:51:04.544255 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:51:04.544255 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:51:04.544255 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 526/1039] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:51:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:51:04.544811 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e6bc3f-57df-4216-a282-2f51df5a3a2d None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:04.570543 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f3ae8c-e1ec-463b-9787-7de5d821c143 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:04.573218 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:04.613913 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386aa7aa-537e-4618-99c9-804d122f7c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:04.614527 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386aa7aa-537e-4618-99c9-804d122f7c8c None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:04.614812 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386aa7aa-537e-4618-99c9-804d122f7c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:04.615089 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386aa7aa-537e-4618-99c9-804d122f7c8c None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:04.615769 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386aa7aa-537e-4618-99c9-804d122f7c8c None admin] RBAC: Adding query filter params (name=image) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:04.616081 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386aa7aa-537e-4618-99c9-804d122f7c8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:04.617470 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386aa7aa-537e-4618-99c9-804d122f7c8c None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:04.623682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 514/1040] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:51:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:51:04.629344 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386aa7aa-537e-4618-99c9-804d122f7c8c None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:04.662458 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e6bc3f-57df-4216-a282-2f51df5a3a2d None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:04.690290 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:04.705261 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d29e62c-f20b-42d6-932d-3d604d62f934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:04.705879 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29e62c-f20b-42d6-932d-3d604d62f934 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:04.706161 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29e62c-f20b-42d6-932d-3d604d62f934 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:04.706440 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29e62c-f20b-42d6-932d-3d604d62f934 None admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:04.707196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d29e62c-f20b-42d6-932d-3d604d62f934 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:04.707492 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d29e62c-f20b-42d6-932d-3d604d62f934 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:04.709001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d29e62c-f20b-42d6-932d-3d604d62f934 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:04.722273 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 527/1041] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:51:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:51:04.728867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d29e62c-f20b-42d6-932d-3d604d62f934 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:04.744048 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386aa7aa-537e-4618-99c9-804d122f7c8c None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:04.779538 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed7f4c4-7ca8-4fee-8627-a1de3fe05c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:04.780150 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed7f4c4-7ca8-4fee-8627-a1de3fe05c28 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:04.780428 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed7f4c4-7ca8-4fee-8627-a1de3fe05c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:04.780743 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed7f4c4-7ca8-4fee-8627-a1de3fe05c28 None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:04.781392 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed7f4c4-7ca8-4fee-8627-a1de3fe05c28 None admin] RBAC: Adding query filter params (type=image) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:04.781696 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed7f4c4-7ca8-4fee-8627-a1de3fe05c28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:04.783040 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed7f4c4-7ca8-4fee-8627-a1de3fe05c28 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:04.788788 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 515/1042] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:51:04 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:51:04.794628 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed7f4c4-7ca8-4fee-8627-a1de3fe05c28 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:04.831510 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:04.855690 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d29e62c-f20b-42d6-932d-3d604d62f934 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:04.892300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6581a4cf-8e0f-44c8-aca8-a832465802ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:04.892965 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581a4cf-8e0f-44c8-aca8-a832465802ce None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:04.893339 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581a4cf-8e0f-44c8-aca8-a832465802ce None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:04.893587 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581a4cf-8e0f-44c8-aca8-a832465802ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:04.894309 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6581a4cf-8e0f-44c8-aca8-a832465802ce None admin] RBAC: Adding query filter params (role.id=0a40666ef2c140d1951dea50f3127507, scope.project.id=80379e2104644e98a2e85d4e4af1b365, user.id=95e9f085e4f04d118373b8abff27543a) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:04.894702 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6581a4cf-8e0f-44c8-aca8-a832465802ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:04.896042 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6581a4cf-8e0f-44c8-aca8-a832465802ce None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:04.902152 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed7f4c4-7ca8-4fee-8627-a1de3fe05c28 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:04.912689 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 528/1043] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:51:04 2022] GET /identity/v3/role_assignments?user.id=95e9f085e4f04d118373b8abff27543a&scope.project.id=80379e2104644e98a2e85d4e4af1b365&role.id=0a40666ef2c140d1951dea50f3127507 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:51:04.934882 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:04.951646 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d506dd6-4908-4f0e-978e-fc29e07f4c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:04.952379 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d506dd6-4908-4f0e-978e-fc29e07f4c5b None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:04.952766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d506dd6-4908-4f0e-978e-fc29e07f4c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:04.953107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d506dd6-4908-4f0e-978e-fc29e07f4c5b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:04.953951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d506dd6-4908-4f0e-978e-fc29e07f4c5b None admin] RBAC: Adding query filter params (interface=public, service_id=82f54ee84723407db0fd3011be10b405, region_id=RegionOne) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:04.954268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d506dd6-4908-4f0e-978e-fc29e07f4c5b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:04.955679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d506dd6-4908-4f0e-978e-fc29e07f4c5b None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:04.965548 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 516/1044] 10.222.0.39 () {56 vars in 1251 bytes} [Fri Aug 5 19:51:04 2022] GET /identity/v3/endpoints?service_id=82f54ee84723407db0fd3011be10b405&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:51:04.968656 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6581a4cf-8e0f-44c8-aca8-a832465802ce None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:05.102444 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6581a4cf-8e0f-44c8-aca8-a832465802ce None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:05.142635 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f6206-d9cb-49a3-aa44-32245ab0dbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:05.142635 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f6206-d9cb-49a3-aa44-32245ab0dbd2 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:05.142635 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f6206-d9cb-49a3-aa44-32245ab0dbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:05.142635 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f6206-d9cb-49a3-aa44-32245ab0dbd2 None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:05.143086 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f6206-d9cb-49a3-aa44-32245ab0dbd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:05.143086 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f6206-d9cb-49a3-aa44-32245ab0dbd2 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:05.155664 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 529/1045] 10.222.0.39 () {56 vars in 1087 bytes} [Fri Aug 5 19:51:04 2022] GET /identity/v3/services => generated 1957 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:51:05.781395 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3393a1-5397-4a2d-b623-920b5bbb26b9 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:05.782009 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3393a1-5397-4a2d-b623-920b5bbb26b9 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:05.782009 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3393a1-5397-4a2d-b623-920b5bbb26b9 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:05.783486 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 517/1046] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:51:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:51:05.788780 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e73a8-87b2-4540-a1bc-cacc48c1c6c3 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:05.789075 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e73a8-87b2-4540-a1bc-cacc48c1c6c3 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:05.789354 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e73a8-87b2-4540-a1bc-cacc48c1c6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:05.809203 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-e21e73a8-87b2-4540-a1bc-cacc48c1c6c3 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:05.810409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e21e73a8-87b2-4540-a1bc-cacc48c1c6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:05.818044 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:05.883489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 530/1047] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:05 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:51:05.908469 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3393a1-5397-4a2d-b623-920b5bbb26b9 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:06.023569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3393a1-5397-4a2d-b623-920b5bbb26b9 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:06.062700 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c4fcc-8bfe-427f-81ba-dba32cc678fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:06.063260 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c4fcc-8bfe-427f-81ba-dba32cc678fb None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:06.063540 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c4fcc-8bfe-427f-81ba-dba32cc678fb None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:06.063812 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c4fcc-8bfe-427f-81ba-dba32cc678fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:06.078449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618c4fcc-8bfe-427f-81ba-dba32cc678fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:51:06.083315 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-618c4fcc-8bfe-427f-81ba-dba32cc678fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:51:06.083315 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:06.083315 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:06.083315 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:06.083315 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:06.083315 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:06.083315 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:06.083315 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:06.083315 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:06.083315 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:06.083315 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:06.083315 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:06.083315 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:51:06.083315 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:51:06.083315 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:51:06.083315 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:51:06.084401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:51:06.084401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:51:06.084401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:51:06.084401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:51:06.084401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:06.084401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:06.084401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:06.084401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:06.084401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:06.084401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:06.084401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:06.084401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:06.084401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:06.084401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:06.084401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:06.084401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:06.092194 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:06.092194 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:06.092194 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:06.092194 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:06.092194 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:51:06.092194 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:51:06.092194 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:51:06.092194 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:51:06.092194 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:51:06.092194 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:51:06.092194 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:51:06.092194 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:51:06.092194 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 518/1048] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:51:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:51:06.094879 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e73a8-87b2-4540-a1bc-cacc48c1c6c3 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:06.220196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e73a8-87b2-4540-a1bc-cacc48c1c6c3 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:06.262755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba060cdd-2400-4adf-933f-5e691c8c4ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:06.263373 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba060cdd-2400-4adf-933f-5e691c8c4ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:06.263649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba060cdd-2400-4adf-933f-5e691c8c4ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:06.263917 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba060cdd-2400-4adf-933f-5e691c8c4ac5 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:06.264654 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba060cdd-2400-4adf-933f-5e691c8c4ac5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:06.264960 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba060cdd-2400-4adf-933f-5e691c8c4ac5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:06.266312 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba060cdd-2400-4adf-933f-5e691c8c4ac5 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:06.276413 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 531/1049] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:51:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:06.282186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c4fcc-8bfe-427f-81ba-dba32cc678fb None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:06.336694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:06.407096 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c4fcc-8bfe-427f-81ba-dba32cc678fb None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:06.444027 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e85125-c25c-4cd9-80b9-60562e079e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:06.444794 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e85125-c25c-4cd9-80b9-60562e079e5b None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:06.445189 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e85125-c25c-4cd9-80b9-60562e079e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:06.445503 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e85125-c25c-4cd9-80b9-60562e079e5b None admin] PATH_INFO: `/v3/users/5bba4c33cbe34e95906c7486676515d7` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:06.462403 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e85125-c25c-4cd9-80b9-60562e079e5b None admin] RBAC: Authorizing `identity:get_user(user_id=5bba4c33cbe34e95906c7486676515d7)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:06.464060 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e85125-c25c-4cd9-80b9-60562e079e5b None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:06.470624 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 519/1050] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:51:06 2022] GET /identity/v3/users/5bba4c33cbe34e95906c7486676515d7 => generated 285 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:06.471604 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba060cdd-2400-4adf-933f-5e691c8c4ac5 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:06.476312 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:06.606856 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba060cdd-2400-4adf-933f-5e691c8c4ac5 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:06.644886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a3fbb8-b182-4f72-a4a2-6904952d725f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:06.645517 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3fbb8-b182-4f72-a4a2-6904952d725f None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:06.645804 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3fbb8-b182-4f72-a4a2-6904952d725f None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:06.646077 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3fbb8-b182-4f72-a4a2-6904952d725f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:06.721546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a3fbb8-b182-4f72-a4a2-6904952d725f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:06.723013 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a3fbb8-b182-4f72-a4a2-6904952d725f None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:06.725754 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:06.840641 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 532/1051] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:51:06 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 19:51:06.843743 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e85125-c25c-4cd9-80b9-60562e079e5b None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:06.951886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19cd33b-2688-4896-9a45-b923a785cc33 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:06.952171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19cd33b-2688-4896-9a45-b923a785cc33 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:06.952449 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19cd33b-2688-4896-9a45-b923a785cc33 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:06.954204 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 533/1052] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:06.959159 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e863915-c518-4968-a9bf-4fd12e72e67f None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:06.959375 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e863915-c518-4968-a9bf-4fd12e72e67f None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:06.959632 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e863915-c518-4968-a9bf-4fd12e72e67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:06.966844 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e85125-c25c-4cd9-80b9-60562e079e5b None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:06.968494 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:06.978943 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-9e863915-c518-4968-a9bf-4fd12e72e67f None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:06.980171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-9e863915-c518-4968-a9bf-4fd12e72e67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:07.010454 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e18f3-092e-4f33-bd16-6cc84999a96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:07.011447 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e18f3-092e-4f33-bd16-6cc84999a96b None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:07.011787 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e18f3-092e-4f33-bd16-6cc84999a96b None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:07.012063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e18f3-092e-4f33-bd16-6cc84999a96b None admin] PATH_INFO: `/v3/projects/b2b9353feedb4cbaada0bc357a07613b` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:07.013602 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e18f3-092e-4f33-bd16-6cc84999a96b None admin] RBAC: Authorizing `identity:get_project(project_id=b2b9353feedb4cbaada0bc357a07613b)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:07.015034 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e18f3-092e-4f33-bd16-6cc84999a96b None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:07.022112 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 520/1053] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:51:06 2022] GET /identity/v3/projects/b2b9353feedb4cbaada0bc357a07613b => generated 311 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:07.023278 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e18f3-092e-4f33-bd16-6cc84999a96b None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:07.064625 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 534/1054] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:51:07.083911 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e863915-c518-4968-a9bf-4fd12e72e67f None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:07.121960 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:07.165818 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e18f3-092e-4f33-bd16-6cc84999a96b None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:07.205018 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e863915-c518-4968-a9bf-4fd12e72e67f None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:07.209226 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75130c-e2e4-4b4f-91b8-0266c7875470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:07.209860 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75130c-e2e4-4b4f-91b8-0266c7875470 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:07.210147 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75130c-e2e4-4b4f-91b8-0266c7875470 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:07.210431 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75130c-e2e4-4b4f-91b8-0266c7875470 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:07.211232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75130c-e2e4-4b4f-91b8-0266c7875470 None admin] RBAC: Adding query filter params (role.id=89919f784ad144529b07c8ab92ee7d48, scope.project.id=b2b9353feedb4cbaada0bc357a07613b, user.id=5bba4c33cbe34e95906c7486676515d7) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:07.211543 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75130c-e2e4-4b4f-91b8-0266c7875470 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:07.214705 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75130c-e2e4-4b4f-91b8-0266c7875470 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:07.228612 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 521/1055] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:51:07 2022] GET /identity/v3/role_assignments?user.id=5bba4c33cbe34e95906c7486676515d7&scope.project.id=b2b9353feedb4cbaada0bc357a07613b&role.id=89919f784ad144529b07c8ab92ee7d48 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:51:07.262673 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1370b2-cc58-44de-8ee1-2dc41881ef87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:07.263412 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1370b2-cc58-44de-8ee1-2dc41881ef87 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:07.263692 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1370b2-cc58-44de-8ee1-2dc41881ef87 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:07.263961 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1370b2-cc58-44de-8ee1-2dc41881ef87 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:07.277768 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1370b2-cc58-44de-8ee1-2dc41881ef87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:51:07.282876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-ab1370b2-cc58-44de-8ee1-2dc41881ef87 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:51:07.282876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:07.282876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:07.282876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:07.282876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:07.282876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:07.282876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:07.282876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:07.282876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:07.282876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:07.282876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:07.282876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:07.282876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:51:07.282876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:51:07.282876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:51:07.282876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:51:07.284068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:51:07.284068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:51:07.284068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:51:07.284068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:51:07.284068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:07.284068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:07.284068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:07.284068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:07.284068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:07.284068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:07.284068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:07.284068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:07.284068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:07.284068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:07.284068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:07.284068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:07.285070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:07.285070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:07.285070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:07.285070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:07.285070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:51:07.285070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:51:07.285070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:51:07.285070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:51:07.285070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:51:07.285070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:51:07.285070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:51:07.285070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:51:07.285070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 535/1056] 10.222.0.39 () {56 vars in 1120 bytes} [Fri Aug 5 19:51:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:51:07.292220 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75130c-e2e4-4b4f-91b8-0266c7875470 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:07.307817 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674dee7-9ac6-4184-a707-1f749332aa61 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:07.307817 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674dee7-9ac6-4184-a707-1f749332aa61 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:07.308136 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674dee7-9ac6-4184-a707-1f749332aa61 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:07.309360 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 536/1057] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:07.316728 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97b1dc-2b74-4048-8818-7e9ff6214293 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:07.317078 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97b1dc-2b74-4048-8818-7e9ff6214293 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:07.317298 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97b1dc-2b74-4048-8818-7e9ff6214293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:07.334576 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:07.339717 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-dc97b1dc-2b74-4048-8818-7e9ff6214293 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:07.340953 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-dc97b1dc-2b74-4048-8818-7e9ff6214293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:07.415262 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75130c-e2e4-4b4f-91b8-0266c7875470 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:07.418267 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 537/1058] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:51:07.440997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97b1dc-2b74-4048-8818-7e9ff6214293 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:07.453462 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6798add8-5903-48eb-ac1f-6e29774d4688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:07.454390 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798add8-5903-48eb-ac1f-6e29774d4688 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:07.454677 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798add8-5903-48eb-ac1f-6e29774d4688 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:07.455210 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798add8-5903-48eb-ac1f-6e29774d4688 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:07.455821 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6798add8-5903-48eb-ac1f-6e29774d4688 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:07.456123 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6798add8-5903-48eb-ac1f-6e29774d4688 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:07.457726 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6798add8-5903-48eb-ac1f-6e29774d4688 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:07.471612 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 522/1059] 10.222.0.39 () {56 vars in 1115 bytes} [Fri Aug 5 19:51:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:07.473316 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6798add8-5903-48eb-ac1f-6e29774d4688 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:07.474155 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:07.558621 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97b1dc-2b74-4048-8818-7e9ff6214293 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:07.585032 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:07.593308 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6798add8-5903-48eb-ac1f-6e29774d4688 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:07.595313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa1db44-d4a3-4dd7-a427-b1e9d584d95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:07.596019 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1db44-d4a3-4dd7-a427-b1e9d584d95e None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:07.596300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1db44-d4a3-4dd7-a427-b1e9d584d95e None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:07.596569 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1db44-d4a3-4dd7-a427-b1e9d584d95e None admin] PATH_INFO: `/v3/services/image` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:07.597406 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa1db44-d4a3-4dd7-a427-b1e9d584d95e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:07.598768 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa1db44-d4a3-4dd7-a427-b1e9d584d95e None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:07.608913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-2aa1db44-d4a3-4dd7-a427-b1e9d584d95e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 19:51:07.608913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:07.608913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:51:07.608913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:51:07.608913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:51:07.608913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:51:07.608913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:51:07.608913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:51:07.608913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 19:51:07.608913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:51:07.608913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:51:07.608913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:51:07.608913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:07.608913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:07.608913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:07.608913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:07.608913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:07.610068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:07.610068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:07.610068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:07.610068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:07.610068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:07.610068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:07.610068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:51:07.610068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:51:07.610068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:51:07.610068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:51:07.610068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:07.610068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:07.610068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:07.610068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:07.610068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:07.610068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:07.613711 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:07.613711 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:07.613711 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:07.613711 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:07.613711 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:07.613711 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:07.613711 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:07.613711 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:07.613711 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:07.613711 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:07.613711 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:51:07.613711 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:51:07.613711 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 19:51:07.613711 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:51:07.613711 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 538/1060] 10.222.0.39 () {56 vars in 1105 bytes} [Fri Aug 5 19:51:07 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:51:07.617055 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa1db44-d4a3-4dd7-a427-b1e9d584d95e None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:07.640284 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe96c9df-b446-4c16-8e0a-e490df4d980d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:07.641190 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96c9df-b446-4c16-8e0a-e490df4d980d None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:07.641681 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96c9df-b446-4c16-8e0a-e490df4d980d None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:07.641958 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96c9df-b446-4c16-8e0a-e490df4d980d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:07.655146 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe96c9df-b446-4c16-8e0a-e490df4d980d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:07.656599 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe96c9df-b446-4c16-8e0a-e490df4d980d None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:07.672945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-fe96c9df-b446-4c16-8e0a-e490df4d980d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:51:07.672945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:07.672945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:07.672945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:07.672945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:07.672945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:07.672945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:07.672945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:07.672945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:07.672945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:07.672945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:07.672945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:07.672945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:51:07.672945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:51:07.672945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:51:07.673931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:51:07.673931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:07.673931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:07.673931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:07.673931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:07.673931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:07.673931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:07.673931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:07.673931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:07.673931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:07.673931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:07.673931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:07.673931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:07.673931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:07.673931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:07.673931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:07.673931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:07.675320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:51:07.675320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:51:07.675320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:51:07.675320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:51:07.675320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 523/1061] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:51:07.679849 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe96c9df-b446-4c16-8e0a-e490df4d980d None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:07.722257 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:07.736819 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa1db44-d4a3-4dd7-a427-b1e9d584d95e None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:07.772748 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbadefdd-ae27-4cf3-9b0f-a482eee8b35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:07.773744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbadefdd-ae27-4cf3-9b0f-a482eee8b35c None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:07.774448 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbadefdd-ae27-4cf3-9b0f-a482eee8b35c None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:07.774766 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbadefdd-ae27-4cf3-9b0f-a482eee8b35c None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:07.775741 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbadefdd-ae27-4cf3-9b0f-a482eee8b35c None admin] RBAC: Adding query filter params (name=image) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:07.776061 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbadefdd-ae27-4cf3-9b0f-a482eee8b35c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:07.777600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbadefdd-ae27-4cf3-9b0f-a482eee8b35c None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:07.787559 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 539/1062] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:51:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:51:07.788797 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbadefdd-ae27-4cf3-9b0f-a482eee8b35c None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:07.799251 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe96c9df-b446-4c16-8e0a-e490df4d980d None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:07.834271 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3001f0f0-d375-4ccf-a3f0-3d94ae232ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:07.834888 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001f0f0-d375-4ccf-a3f0-3d94ae232ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:07.835167 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001f0f0-d375-4ccf-a3f0-3d94ae232ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:07.835440 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001f0f0-d375-4ccf-a3f0-3d94ae232ca9 None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:07.836129 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3001f0f0-d375-4ccf-a3f0-3d94ae232ca9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:07.836433 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3001f0f0-d375-4ccf-a3f0-3d94ae232ca9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:07.837959 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3001f0f0-d375-4ccf-a3f0-3d94ae232ca9 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:07.851143 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 524/1063] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:51:07.857283 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3001f0f0-d375-4ccf-a3f0-3d94ae232ca9 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:07.868333 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:07.904201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbadefdd-ae27-4cf3-9b0f-a482eee8b35c None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:07.942808 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe26492-4ac5-4097-a6a5-5a3e43cff42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:07.943524 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe26492-4ac5-4097-a6a5-5a3e43cff42c None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:07.943810 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe26492-4ac5-4097-a6a5-5a3e43cff42c None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:07.944087 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe26492-4ac5-4097-a6a5-5a3e43cff42c None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:07.945133 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe26492-4ac5-4097-a6a5-5a3e43cff42c None admin] RBAC: Adding query filter params (type=image) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:07.945408 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe26492-4ac5-4097-a6a5-5a3e43cff42c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:07.947190 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe26492-4ac5-4097-a6a5-5a3e43cff42c None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:07.957348 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 540/1064] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:51:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:51:07.958383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe26492-4ac5-4097-a6a5-5a3e43cff42c None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:07.978030 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3001f0f0-d375-4ccf-a3f0-3d94ae232ca9 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:07.980639 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:08.017495 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da087a1-ec92-4759-9b38-e881855eb4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:08.018142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da087a1-ec92-4759-9b38-e881855eb4a7 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:08.018424 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da087a1-ec92-4759-9b38-e881855eb4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:08.018700 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da087a1-ec92-4759-9b38-e881855eb4a7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:08.027594 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da087a1-ec92-4759-9b38-e881855eb4a7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:08.029081 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da087a1-ec92-4759-9b38-e881855eb4a7 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:08.042087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-5da087a1-ec92-4759-9b38-e881855eb4a7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:51:08.042087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:08.042087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:08.042087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:08.042087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:08.042087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:08.042087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:08.042087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:08.042087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:08.042087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:08.042087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:08.042087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:08.042087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:51:08.042087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:51:08.042087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:51:08.042087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:51:08.043148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:08.043148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:08.043148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:51:08.043148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:51:08.043148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:51:08.043148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:51:08.043148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:08.043148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:08.043148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:08.043148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:08.043148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:08.043148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:08.043148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:08.043148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:08.043148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:08.043148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:08.043148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:08.043148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:08.045428 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:08.045428 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:08.045428 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:51:08.045428 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:51:08.045428 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:51:08.045428 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:51:08.045428 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:51:08.045428 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:51:08.045428 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:51:08.045428 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:51:08.045428 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 525/1065] 10.222.0.39 () {56 vars in 1093 bytes} [Fri Aug 5 19:51:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:51:08.049246 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da087a1-ec92-4759-9b38-e881855eb4a7 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:08.080499 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe26492-4ac5-4097-a6a5-5a3e43cff42c None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:08.122502 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ce925-c3e0-4b90-b71e-9eb51d64c74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:08.123128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ce925-c3e0-4b90-b71e-9eb51d64c74e None admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:08.123413 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ce925-c3e0-4b90-b71e-9eb51d64c74e None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:08.123689 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ce925-c3e0-4b90-b71e-9eb51d64c74e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:08.124414 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ce925-c3e0-4b90-b71e-9eb51d64c74e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:08.125925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ce925-c3e0-4b90-b71e-9eb51d64c74e None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:08.128110 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:08.150284 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 541/1066] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 19:51:07 2022] POST /identity/v3/endpoints => generated 327 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:51:08.176908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da087a1-ec92-4759-9b38-e881855eb4a7 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:08.225270 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34fea4-dc7d-4007-865b-98639a5246e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:08.225899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34fea4-dc7d-4007-865b-98639a5246e6 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:08.226185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34fea4-dc7d-4007-865b-98639a5246e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:08.226463 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34fea4-dc7d-4007-865b-98639a5246e6 None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:08.228192 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e34fea4-dc7d-4007-865b-98639a5246e6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:08.228318 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e34fea4-dc7d-4007-865b-98639a5246e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:08.229834 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e34fea4-dc7d-4007-865b-98639a5246e6 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:08.258063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 526/1067] 10.222.0.39 () {56 vars in 1133 bytes} [Fri Aug 5 19:51:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:51:08.259492 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ce925-c3e0-4b90-b71e-9eb51d64c74e None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:08.343439 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:08.381366 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ce925-c3e0-4b90-b71e-9eb51d64c74e None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:08.420413 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee96733-8a8b-4865-a7dc-a994fe723704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:08.421090 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee96733-8a8b-4865-a7dc-a994fe723704 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:08.421367 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee96733-8a8b-4865-a7dc-a994fe723704 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:08.421642 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee96733-8a8b-4865-a7dc-a994fe723704 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:08.436555 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee96733-8a8b-4865-a7dc-a994fe723704 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:08.438044 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee96733-8a8b-4865-a7dc-a994fe723704 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:08.453343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-0ee96733-8a8b-4865-a7dc-a994fe723704 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:51:08.453343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:08.453343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:08.453343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:08.453343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:08.453343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:08.453343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:08.453343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:08.453343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:08.453343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:08.453343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:08.453343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:08.453343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:51:08.453343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:51:08.453343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:51:08.454501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:51:08.454501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:08.454501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:08.454501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:08.454501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:08.454501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:08.454501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:08.454501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:08.454501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:08.454501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:08.454501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:08.454501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:08.454501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:08.454501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:08.454501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:08.454501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:08.454501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:08.455454 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:51:08.455454 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:51:08.455454 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:51:08.455454 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:51:08.455454 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 542/1068] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:51:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:51:08.462682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34fea4-dc7d-4007-865b-98639a5246e6 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:08.583755 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34fea4-dc7d-4007-865b-98639a5246e6 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:08.611517 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:08.627295 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec09b8-6c7c-4dad-94f8-dd75594a7f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:08.627928 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec09b8-6c7c-4dad-94f8-dd75594a7f72 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:08.628252 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec09b8-6c7c-4dad-94f8-dd75594a7f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:08.628556 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec09b8-6c7c-4dad-94f8-dd75594a7f72 None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:08.629466 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec09b8-6c7c-4dad-94f8-dd75594a7f72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:08.629767 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec09b8-6c7c-4dad-94f8-dd75594a7f72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:08.631109 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec09b8-6c7c-4dad-94f8-dd75594a7f72 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:08.651876 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 527/1069] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:08.653617 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee96733-8a8b-4865-a7dc-a994fe723704 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:08.768833 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee96733-8a8b-4865-a7dc-a994fe723704 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:08.794966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:08.811925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcb66f-7948-422c-ba63-0a2af097c8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:08.812568 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcb66f-7948-422c-ba63-0a2af097c8e7 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:08.812878 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcb66f-7948-422c-ba63-0a2af097c8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:08.813152 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcb66f-7948-422c-ba63-0a2af097c8e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:08.821281 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dcb66f-7948-422c-ba63-0a2af097c8e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:08.822913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dcb66f-7948-422c-ba63-0a2af097c8e7 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:08.835634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-a5dcb66f-7948-422c-ba63-0a2af097c8e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:51:08.835634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:08.835634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:08.835634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:08.835634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:08.835634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:08.835634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:08.835634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:08.835634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:08.835634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:08.835634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:08.835634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:08.835634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:51:08.835634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:51:08.835634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:51:08.836620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:51:08.836620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:08.836620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:08.836620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:08.836620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:08.836620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:08.836620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:08.836620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:08.836620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:08.836620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:08.836620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:08.836620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:08.836620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:08.836620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:08.836620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:08.836620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:08.836620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:08.837794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:51:08.837794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:51:08.837794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:51:08.837794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:51:08.837794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:51:08.837794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:51:08.837794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:51:08.837794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:51:08.840964 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 543/1070] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:51:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:51:08.843373 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec09b8-6c7c-4dad-94f8-dd75594a7f72 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:08.956302 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec09b8-6c7c-4dad-94f8-dd75594a7f72 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:08.993864 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3353b0de-61d6-4911-abe2-ee7cb47bbf4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:08.994596 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353b0de-61d6-4911-abe2-ee7cb47bbf4d None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:08.994785 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353b0de-61d6-4911-abe2-ee7cb47bbf4d None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:08.995052 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353b0de-61d6-4911-abe2-ee7cb47bbf4d None admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:08.995850 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3353b0de-61d6-4911-abe2-ee7cb47bbf4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:08.996151 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3353b0de-61d6-4911-abe2-ee7cb47bbf4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:08.997697 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3353b0de-61d6-4911-abe2-ee7cb47bbf4d None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:09.015712 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 528/1071] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:51:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:09.018542 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcb66f-7948-422c-ba63-0a2af097c8e7 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:09.136981 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcb66f-7948-422c-ba63-0a2af097c8e7 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:09.183777 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af574a4-9211-4697-87b5-96cec130d0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:09.184448 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af574a4-9211-4697-87b5-96cec130d0cb None admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:09.184763 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af574a4-9211-4697-87b5-96cec130d0cb None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:09.185041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af574a4-9211-4697-87b5-96cec130d0cb None admin] PATH_INFO: `/v3/projects/80379e2104644e98a2e85d4e4af1b365/users/95e9f085e4f04d118373b8abff27543a/roles/0a40666ef2c140d1951dea50f3127507` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:09.187291 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af574a4-9211-4697-87b5-96cec130d0cb None admin] RBAC: Authorizing `identity:create_grant(project_id=80379e2104644e98a2e85d4e4af1b365, user_id=95e9f085e4f04d118373b8abff27543a, role_id=0a40666ef2c140d1951dea50f3127507)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:09.188753 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af574a4-9211-4697-87b5-96cec130d0cb None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:09.204064 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 544/1072] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Aug 5 19:51:09 2022] PUT /identity/v3/projects/80379e2104644e98a2e85d4e4af1b365/users/95e9f085e4f04d118373b8abff27543a/roles/0a40666ef2c140d1951dea50f3127507 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:09.265697 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485fe89b-f6cd-4a2b-8605-eef1c1dbd378 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:09.266019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485fe89b-f6cd-4a2b-8605-eef1c1dbd378 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:09.266253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485fe89b-f6cd-4a2b-8605-eef1c1dbd378 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:09.267628 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 529/1073] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:51:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:09.273802 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818ceab-4a95-49d7-9a69-0d258c35fa93 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:09.274191 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818ceab-4a95-49d7-9a69-0d258c35fa93 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:09.274937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818ceab-4a95-49d7-9a69-0d258c35fa93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:09.295317 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-3818ceab-4a95-49d7-9a69-0d258c35fa93 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:09.296527 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-3818ceab-4a95-49d7-9a69-0d258c35fa93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:09.354677 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:09.392789 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 545/1074] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:51:09.417423 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485fe89b-f6cd-4a2b-8605-eef1c1dbd378 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:09.468928 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:09.533444 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485fe89b-f6cd-4a2b-8605-eef1c1dbd378 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:09.571921 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc0303-514c-45b6-83cf-616d6ff401c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:09.572487 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc0303-514c-45b6-83cf-616d6ff401c6 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:09.572804 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc0303-514c-45b6-83cf-616d6ff401c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:09.573083 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc0303-514c-45b6-83cf-616d6ff401c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:09.584664 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cc0303-514c-45b6-83cf-616d6ff401c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:51:09.592062 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-f0cc0303-514c-45b6-83cf-616d6ff401c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:51:09.592062 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:09.592062 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:09.592062 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:09.592062 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:09.592062 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:09.592062 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:09.592062 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:09.592062 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:09.592062 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:09.592062 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:09.592062 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:09.592062 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:51:09.592062 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:51:09.592062 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:51:09.592062 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:51:09.593105 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:51:09.593105 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:51:09.593105 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:51:09.593105 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:51:09.593105 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:09.593105 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:09.593105 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:09.593105 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:09.593105 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:09.593105 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:09.593105 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:09.593105 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:09.593105 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:09.593105 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:09.593105 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:09.593105 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:09.594029 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:09.594029 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:09.594029 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:09.594029 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:09.594029 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:51:09.594029 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:51:09.594029 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:51:09.594029 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:51:09.594029 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:51:09.594029 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:51:09.594029 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:51:09.594029 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:51:09.594029 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 530/1075] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:51:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:51:09.601961 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3818ceab-4a95-49d7-9a69-0d258c35fa93 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:09.718028 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3818ceab-4a95-49d7-9a69-0d258c35fa93 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:09.763093 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4126d61-39f6-41dd-ab58-59fefab856c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:09.763718 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4126d61-39f6-41dd-ab58-59fefab856c0 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:09.764002 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4126d61-39f6-41dd-ab58-59fefab856c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:09.764276 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4126d61-39f6-41dd-ab58-59fefab856c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:09.765040 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4126d61-39f6-41dd-ab58-59fefab856c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:09.765346 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4126d61-39f6-41dd-ab58-59fefab856c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:09.766709 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4126d61-39f6-41dd-ab58-59fefab856c0 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:09.782429 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 546/1076] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:51:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:51:09.783625 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc0303-514c-45b6-83cf-616d6ff401c6 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:09.893763 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc0303-514c-45b6-83cf-616d6ff401c6 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:09.895457 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:09.936080 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0398e-3d77-41c1-b0bf-7b9cc8689b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:09.936777 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0398e-3d77-41c1-b0bf-7b9cc8689b4f None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:09.937056 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0398e-3d77-41c1-b0bf-7b9cc8689b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:09.937334 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0398e-3d77-41c1-b0bf-7b9cc8689b4f None admin] PATH_INFO: `/v3/users/5bba4c33cbe34e95906c7486676515d7` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:09.938551 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe0398e-3d77-41c1-b0bf-7b9cc8689b4f None admin] RBAC: Authorizing `identity:get_user(user_id=5bba4c33cbe34e95906c7486676515d7)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:09.940014 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe0398e-3d77-41c1-b0bf-7b9cc8689b4f None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:09.947159 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 531/1077] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:51:09 2022] GET /identity/v3/users/5bba4c33cbe34e95906c7486676515d7 => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:09.948276 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4126d61-39f6-41dd-ab58-59fefab856c0 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:09.988618 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:10.067754 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4126d61-39f6-41dd-ab58-59fefab856c0 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:10.103228 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5668a7a-3056-4d22-9cc4-bd918bbb5784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:10.103867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5668a7a-3056-4d22-9cc4-bd918bbb5784 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:10.104153 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5668a7a-3056-4d22-9cc4-bd918bbb5784 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:10.104427 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5668a7a-3056-4d22-9cc4-bd918bbb5784 None admin] PATH_INFO: `/v3/projects/b2b9353feedb4cbaada0bc357a07613b` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:10.105644 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5668a7a-3056-4d22-9cc4-bd918bbb5784 None admin] RBAC: Authorizing `identity:get_project(project_id=b2b9353feedb4cbaada0bc357a07613b)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:10.107077 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5668a7a-3056-4d22-9cc4-bd918bbb5784 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:10.113862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 547/1078] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:51:09 2022] GET /identity/v3/projects/b2b9353feedb4cbaada0bc357a07613b => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:10.116144 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0398e-3d77-41c1-b0bf-7b9cc8689b4f None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:10.228309 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0398e-3d77-41c1-b0bf-7b9cc8689b4f None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:10.255031 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4882e1d-d162-4a92-a951-11be17cc9624 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:10.255268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4882e1d-d162-4a92-a951-11be17cc9624 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:10.255509 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4882e1d-d162-4a92-a951-11be17cc9624 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:10.256954 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 548/1079] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:10.262735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58abb1fc-0c9e-4baf-9842-6721e8d6d7b0 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:10.263016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58abb1fc-0c9e-4baf-9842-6721e8d6d7b0 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:10.263291 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58abb1fc-0c9e-4baf-9842-6721e8d6d7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:10.267940 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661cf39-7040-4029-9840-fe8f28355883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:10.268625 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661cf39-7040-4029-9840-fe8f28355883 None admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:10.268902 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661cf39-7040-4029-9840-fe8f28355883 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:10.269174 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661cf39-7040-4029-9840-fe8f28355883 None admin] PATH_INFO: `/v3/projects/b2b9353feedb4cbaada0bc357a07613b/users/5bba4c33cbe34e95906c7486676515d7/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:10.271400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9661cf39-7040-4029-9840-fe8f28355883 None admin] RBAC: Authorizing `identity:create_grant(project_id=b2b9353feedb4cbaada0bc357a07613b, user_id=5bba4c33cbe34e95906c7486676515d7, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:10.273120 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9661cf39-7040-4029-9840-fe8f28355883 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:10.283296 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-58abb1fc-0c9e-4baf-9842-6721e8d6d7b0 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:10.284482 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-58abb1fc-0c9e-4baf-9842-6721e8d6d7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:10.296643 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 532/1080] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Aug 5 19:51:10 2022] PUT /identity/v3/projects/b2b9353feedb4cbaada0bc357a07613b/users/5bba4c33cbe34e95906c7486676515d7/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:10.373248 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 549/1081] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:51:10.397677 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661cf39-7040-4029-9840-fe8f28355883 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:10.433538 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:10.520239 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661cf39-7040-4029-9840-fe8f28355883 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:10.557999 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe5766-415f-4b86-a366-5d5d9f62ac71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:10.558720 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe5766-415f-4b86-a366-5d5d9f62ac71 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:10.559064 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe5766-415f-4b86-a366-5d5d9f62ac71 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:10.559403 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe5766-415f-4b86-a366-5d5d9f62ac71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:10.630672 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fe5766-415f-4b86-a366-5d5d9f62ac71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:10.632268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fe5766-415f-4b86-a366-5d5d9f62ac71 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:10.663548 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:10.750126 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 533/1082] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:51:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:51:10.755833 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58abb1fc-0c9e-4baf-9842-6721e8d6d7b0 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:10.794723 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:10.866587 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58abb1fc-0c9e-4baf-9842-6721e8d6d7b0 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:10.902947 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a943c-df97-4917-b2dc-f3befdff9c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:10.903580 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a943c-df97-4917-b2dc-f3befdff9c91 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:10.903862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a943c-df97-4917-b2dc-f3befdff9c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:10.904190 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a943c-df97-4917-b2dc-f3befdff9c91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:10.917685 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88a943c-df97-4917-b2dc-f3befdff9c91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:10.919149 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88a943c-df97-4917-b2dc-f3befdff9c91 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:10.935196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-b88a943c-df97-4917-b2dc-f3befdff9c91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:51:10.935196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:10.935196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:10.935196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:10.935196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:10.935196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:10.935196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:10.935196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:10.935196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:10.935196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:10.935196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:10.935196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:10.935196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:51:10.935196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:51:10.935196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:51:10.936173 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:51:10.936173 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:10.936173 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:10.936173 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:10.936173 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:10.936173 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:10.936173 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:10.936173 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:10.936173 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:10.936173 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:10.936173 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:10.936173 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:10.936173 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:10.936173 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:10.936173 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:10.936173 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:10.936173 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:10.937664 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:51:10.937664 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:51:10.937664 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:51:10.937664 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:51:10.937664 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 550/1083] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:51:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:51:10.942342 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe5766-415f-4b86-a366-5d5d9f62ac71 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:11.051713 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe5766-415f-4b86-a366-5d5d9f62ac71 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:11.085832 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:11.093293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5391b-d267-4f1e-9e86-90734eed9932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:11.093921 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5391b-d267-4f1e-9e86-90734eed9932 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:11.094214 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5391b-d267-4f1e-9e86-90734eed9932 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:11.094467 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5391b-d267-4f1e-9e86-90734eed9932 None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:11.095168 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad5391b-d267-4f1e-9e86-90734eed9932 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:11.095473 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad5391b-d267-4f1e-9e86-90734eed9932 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:11.096902 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad5391b-d267-4f1e-9e86-90734eed9932 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:11.109796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 534/1084] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:11.293187 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a4a8a-d08b-442c-93b1-c0de5f7398e1 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:11.293475 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a4a8a-d08b-442c-93b1-c0de5f7398e1 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:11.293747 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a4a8a-d08b-442c-93b1-c0de5f7398e1 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:11.295102 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 551/1085] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:11.300694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f37d0-b3af-4bb1-8525-095f9ff64524 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:11.301136 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f37d0-b3af-4bb1-8525-095f9ff64524 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:11.301318 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f37d0-b3af-4bb1-8525-095f9ff64524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:11.322290 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-d76f37d0-b3af-4bb1-8525-095f9ff64524 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:11.323410 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-d76f37d0-b3af-4bb1-8525-095f9ff64524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:11.400764 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 535/1086] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:51:11.424021 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a4a8a-d08b-442c-93b1-c0de5f7398e1 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:11.514097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:11.530632 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a4a8a-d08b-442c-93b1-c0de5f7398e1 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:11.565056 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb0b8b-927b-4f7c-a1af-6319cbc47941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:11.566138 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb0b8b-927b-4f7c-a1af-6319cbc47941 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:11.566138 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb0b8b-927b-4f7c-a1af-6319cbc47941 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:11.566354 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb0b8b-927b-4f7c-a1af-6319cbc47941 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:11.576920 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb0b8b-927b-4f7c-a1af-6319cbc47941 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:51:11.582683 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-8ddb0b8b-927b-4f7c-a1af-6319cbc47941 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:51:11.582683 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:11.582683 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:11.582683 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:11.582683 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:11.582683 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:11.582683 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:11.582683 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:11.582683 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:11.582683 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:11.582683 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:11.582683 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:11.582683 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:51:11.582683 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:51:11.582683 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:51:11.582683 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:51:11.583802 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:51:11.583802 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:51:11.583802 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:51:11.583802 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:51:11.583802 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:11.583802 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:11.583802 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:11.583802 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:11.583802 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:11.583802 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:11.583802 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:11.583802 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:11.583802 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:11.583802 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:11.583802 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:11.583802 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:11.584907 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:11.584907 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:11.584907 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:11.584907 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:11.584907 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:51:11.584907 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:51:11.584907 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:51:11.584907 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:51:11.584907 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:51:11.584907 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:51:11.584907 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:51:11.584907 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:51:11.584907 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 552/1087] 10.222.0.39 () {56 vars in 1120 bytes} [Fri Aug 5 19:51:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:51:11.589326 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f37d0-b3af-4bb1-8525-095f9ff64524 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:11.703882 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f37d0-b3af-4bb1-8525-095f9ff64524 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:11.730744 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:11.748897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f169fc-3c86-48cd-8ed3-dcc37b987dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:11.748897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f169fc-3c86-48cd-8ed3-dcc37b987dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:11.748897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f169fc-3c86-48cd-8ed3-dcc37b987dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:11.748897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f169fc-3c86-48cd-8ed3-dcc37b987dc4 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:11.749349 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f169fc-3c86-48cd-8ed3-dcc37b987dc4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:11.749349 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f169fc-3c86-48cd-8ed3-dcc37b987dc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:11.750157 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f169fc-3c86-48cd-8ed3-dcc37b987dc4 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:11.760118 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 536/1088] 10.222.0.39 () {56 vars in 1115 bytes} [Fri Aug 5 19:51:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:11.765970 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb0b8b-927b-4f7c-a1af-6319cbc47941 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:11.876104 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb0b8b-927b-4f7c-a1af-6319cbc47941 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:11.911110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c8e0f-aeae-4186-867b-5210f9b1aeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:11.911743 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c8e0f-aeae-4186-867b-5210f9b1aeb1 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:11.912027 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c8e0f-aeae-4186-867b-5210f9b1aeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:11.912303 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c8e0f-aeae-4186-867b-5210f9b1aeb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:11.923042 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c8e0f-aeae-4186-867b-5210f9b1aeb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:11.924487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c8e0f-aeae-4186-867b-5210f9b1aeb1 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:11.931311 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:11.939941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-e66c8e0f-aeae-4186-867b-5210f9b1aeb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:51:11.939941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:11.939941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:11.939941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:11.939941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:11.939941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:11.939941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:11.939941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:11.939941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:11.939941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:11.939941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:11.939941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:11.939941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:51:11.939941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:51:11.939941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:51:11.940893 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:51:11.940893 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:11.940893 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:11.940893 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:11.940893 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:11.940893 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:11.940893 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:11.940893 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:11.940893 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:11.940893 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:11.940893 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:11.940893 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:11.940893 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:11.940893 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:11.940893 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:11.940893 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:11.940893 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:11.941832 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:51:11.941832 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:51:11.941832 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:51:11.941832 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:51:11.941832 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 553/1089] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:51:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:51:11.947595 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f169fc-3c86-48cd-8ed3-dcc37b987dc4 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:12.070387 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f169fc-3c86-48cd-8ed3-dcc37b987dc4 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:12.112269 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5486e-dc48-41ad-8f0e-bbe204a679ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:12.112948 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5486e-dc48-41ad-8f0e-bbe204a679ff None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:12.113283 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5486e-dc48-41ad-8f0e-bbe204a679ff None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:12.113622 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5486e-dc48-41ad-8f0e-bbe204a679ff None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:12.114340 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d5486e-dc48-41ad-8f0e-bbe204a679ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:12.114651 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d5486e-dc48-41ad-8f0e-bbe204a679ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:12.116139 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d5486e-dc48-41ad-8f0e-bbe204a679ff None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:12.131713 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 537/1090] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:51:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:12.140497 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c8e0f-aeae-4186-867b-5210f9b1aeb1 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:12.256300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c8e0f-aeae-4186-867b-5210f9b1aeb1 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:12.300552 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77deb653-4e2d-4900-aed1-f8bc73568acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:12.301692 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77deb653-4e2d-4900-aed1-f8bc73568acb None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:12.302174 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77deb653-4e2d-4900-aed1-f8bc73568acb None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:12.302643 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77deb653-4e2d-4900-aed1-f8bc73568acb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:12.317563 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77deb653-4e2d-4900-aed1-f8bc73568acb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:12.320260 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77deb653-4e2d-4900-aed1-f8bc73568acb None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:12.342792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-77deb653-4e2d-4900-aed1-f8bc73568acb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:51:12.342792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:12.342792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:12.342792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:12.342792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:12.342792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:12.342792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:12.342792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:12.342792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:12.342792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:12.342792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:12.342792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:12.342792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:51:12.342792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:51:12.342792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:51:12.342792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:51:12.343846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:12.343846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:12.343846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:51:12.343846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:51:12.343846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:51:12.343846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:51:12.343846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:12.343846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:12.343846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:12.343846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:12.343846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:12.343846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:12.343846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:12.343846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:12.343846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:12.343846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:12.343846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:12.343846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:12.344843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:12.344843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:12.344843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:51:12.344843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:51:12.344843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:51:12.344843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:51:12.344843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:51:12.344843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:51:12.344843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:51:12.344843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:51:12.345389 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 554/1091] 10.222.0.39 () {56 vars in 1093 bytes} [Fri Aug 5 19:51:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:51:12.351073 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5486e-dc48-41ad-8f0e-bbe204a679ff None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:12.402402 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:12.450729 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab627df0-3141-4639-90b7-fe5544f9b3e0 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:12.451251 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab627df0-3141-4639-90b7-fe5544f9b3e0 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:12.451251 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab627df0-3141-4639-90b7-fe5544f9b3e0 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:12.452611 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 555/1092] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:51:12.457527 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b0e50-17d5-424c-83f5-ecb676eafc84 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:12.458022 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b0e50-17d5-424c-83f5-ecb676eafc84 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:12.458022 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b0e50-17d5-424c-83f5-ecb676eafc84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:12.476069 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-078b0e50-17d5-424c-83f5-ecb676eafc84 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:12.476281 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5486e-dc48-41ad-8f0e-bbe204a679ff None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:12.477959 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-078b0e50-17d5-424c-83f5-ecb676eafc84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:12.517447 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce8c0c-054c-485d-90e8-42ac97f7fe7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:12.518070 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce8c0c-054c-485d-90e8-42ac97f7fe7c None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:12.518350 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce8c0c-054c-485d-90e8-42ac97f7fe7c None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:12.518625 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce8c0c-054c-485d-90e8-42ac97f7fe7c None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:12.519425 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce8c0c-054c-485d-90e8-42ac97f7fe7c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:12.519736 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce8c0c-054c-485d-90e8-42ac97f7fe7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:12.521383 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce8c0c-054c-485d-90e8-42ac97f7fe7c None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:12.544141 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 538/1093] 10.222.0.39 () {56 vars in 1133 bytes} [Fri Aug 5 19:51:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:51:12.545266 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce8c0c-054c-485d-90e8-42ac97f7fe7c None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:12.546089 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:12.560754 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 556/1094] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:51:12.583623 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b0e50-17d5-424c-83f5-ecb676eafc84 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:12.650177 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:12.653270 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce8c0c-054c-485d-90e8-42ac97f7fe7c None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:12.688898 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9aaa6c-13f5-40d0-a6c1-362c856965cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:12.689370 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b0e50-17d5-424c-83f5-ecb676eafc84 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:12.689553 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9aaa6c-13f5-40d0-a6c1-362c856965cb None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:12.689796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9aaa6c-13f5-40d0-a6c1-362c856965cb None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:12.690070 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9aaa6c-13f5-40d0-a6c1-362c856965cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:12.724548 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aff338-61d4-435a-b1e2-3a666b0e6cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:12.725206 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aff338-61d4-435a-b1e2-3a666b0e6cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:12.725487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aff338-61d4-435a-b1e2-3a666b0e6cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:12.725762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aff338-61d4-435a-b1e2-3a666b0e6cc9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:12.736923 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aff338-61d4-435a-b1e2-3a666b0e6cc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:51:12.741826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-c8aff338-61d4-435a-b1e2-3a666b0e6cc9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:51:12.741826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:12.741826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:12.741826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:12.741826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:12.741826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:12.741826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:12.741826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:12.741826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:12.741826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:12.741826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:12.741826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:12.741826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:51:12.741826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:51:12.741826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:51:12.741826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:51:12.742986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:51:12.742986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:51:12.742986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:51:12.742986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:51:12.742986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:12.742986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:12.742986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:12.742986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:12.742986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:12.742986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:12.742986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:12.742986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:12.742986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:12.742986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:12.742986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:12.742986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:12.744029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:12.744029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:12.744029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:12.744029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:12.744029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:51:12.744029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:51:12.744029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:51:12.744029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:51:12.744029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:51:12.744029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:51:12.744029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:51:12.744029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:51:12.744029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 557/1095] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:51:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:51:12.749506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aff338-61d4-435a-b1e2-3a666b0e6cc9 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:12.760558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9aaa6c-13f5-40d0-a6c1-362c856965cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:12.762035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9aaa6c-13f5-40d0-a6c1-362c856965cb None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:12.800400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:12.851426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aff338-61d4-435a-b1e2-3a666b0e6cc9 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:12.859071 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:12.863271 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 539/1096] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:51:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:51:12.868840 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9aaa6c-13f5-40d0-a6c1-362c856965cb None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:12.886590 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f98b9-1cfc-4b8c-8065-56ec77c989d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:12.887211 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f98b9-1cfc-4b8c-8065-56ec77c989d1 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:12.887491 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f98b9-1cfc-4b8c-8065-56ec77c989d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:12.887765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f98b9-1cfc-4b8c-8065-56ec77c989d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:12.888471 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f98b9-1cfc-4b8c-8065-56ec77c989d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:12.888805 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f98b9-1cfc-4b8c-8065-56ec77c989d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:12.890165 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f98b9-1cfc-4b8c-8065-56ec77c989d1 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:12.898534 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 558/1097] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:51:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:51:12.904972 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f98b9-1cfc-4b8c-8065-56ec77c989d1 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:12.971670 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9aaa6c-13f5-40d0-a6c1-362c856965cb None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:13.004793 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f98b9-1cfc-4b8c-8065-56ec77c989d1 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:13.006572 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c90918a-e132-4a56-aeae-77469aefad46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:13.007217 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90918a-e132-4a56-aeae-77469aefad46 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:13.007501 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90918a-e132-4a56-aeae-77469aefad46 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:13.007772 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90918a-e132-4a56-aeae-77469aefad46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:13.015934 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:13.019397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c90918a-e132-4a56-aeae-77469aefad46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:13.020905 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c90918a-e132-4a56-aeae-77469aefad46 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:13.036642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-9c90918a-e132-4a56-aeae-77469aefad46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:51:13.036642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:13.036642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:13.036642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:13.036642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:13.036642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:13.036642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:13.036642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:13.036642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:13.036642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:13.036642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:13.036642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:13.036642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:51:13.036642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:51:13.036642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:51:13.037569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:51:13.037569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:13.037569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:13.037569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:13.037569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:13.037569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:13.037569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:13.037569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:13.037569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:13.037569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:13.037569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:13.037569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:13.037569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:13.037569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:13.037569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:13.037569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:13.037569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:13.038474 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:51:13.038474 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:51:13.038474 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:51:13.038474 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:51:13.038474 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 540/1098] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:51:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:51:13.040874 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a1d58-1bed-4af0-8bdf-e2ddc3057ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:13.041509 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a1d58-1bed-4af0-8bdf-e2ddc3057ccb None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:13.041792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a1d58-1bed-4af0-8bdf-e2ddc3057ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:13.042066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a1d58-1bed-4af0-8bdf-e2ddc3057ccb None admin] PATH_INFO: `/v3/users/5bba4c33cbe34e95906c7486676515d7` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:13.043336 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134a1d58-1bed-4af0-8bdf-e2ddc3057ccb None admin] RBAC: Authorizing `identity:get_user(user_id=5bba4c33cbe34e95906c7486676515d7)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:13.044007 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c90918a-e132-4a56-aeae-77469aefad46 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:13.044794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134a1d58-1bed-4af0-8bdf-e2ddc3057ccb None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:13.046408 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 559/1099] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:51:12 2022] GET /identity/v3/users/5bba4c33cbe34e95906c7486676515d7 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:51:13.051849 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a1d58-1bed-4af0-8bdf-e2ddc3057ccb None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:13.099762 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:13.150500 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c90918a-e132-4a56-aeae-77469aefad46 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:13.157593 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a1d58-1bed-4af0-8bdf-e2ddc3057ccb None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:13.184788 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a13a4-077d-48c0-914a-acee204dc78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:13.185399 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a13a4-077d-48c0-914a-acee204dc78e None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:13.185688 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a13a4-077d-48c0-914a-acee204dc78e None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:13.185965 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a13a4-077d-48c0-914a-acee204dc78e None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:13.186654 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a13a4-077d-48c0-914a-acee204dc78e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:13.186963 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a13a4-077d-48c0-914a-acee204dc78e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:13.188314 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a13a4-077d-48c0-914a-acee204dc78e None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:13.191205 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb5f70-bd53-4d86-9fd2-c18de4ce5850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:13.191818 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb5f70-bd53-4d86-9fd2-c18de4ce5850 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:13.192101 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb5f70-bd53-4d86-9fd2-c18de4ce5850 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:13.192374 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb5f70-bd53-4d86-9fd2-c18de4ce5850 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:13.200860 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 541/1100] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:51:13.206294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a13a4-077d-48c0-914a-acee204dc78e None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:13.241266 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:13.266819 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb5f70-bd53-4d86-9fd2-c18de4ce5850 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:13.268267 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb5f70-bd53-4d86-9fd2-c18de4ce5850 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:13.310814 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a13a4-077d-48c0-914a-acee204dc78e None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:13.324828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:13.347870 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54926c5f-99ee-4236-b4d4-5eb23049b2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:13.348509 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54926c5f-99ee-4236-b4d4-5eb23049b2fb None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:13.348823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54926c5f-99ee-4236-b4d4-5eb23049b2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:13.349103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54926c5f-99ee-4236-b4d4-5eb23049b2fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:13.357295 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54926c5f-99ee-4236-b4d4-5eb23049b2fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:13.358968 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54926c5f-99ee-4236-b4d4-5eb23049b2fb None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:13.373203 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-54926c5f-99ee-4236-b4d4-5eb23049b2fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:51:13.373203 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:13.373203 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:13.373203 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:13.373203 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:13.373203 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:13.373203 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:13.373203 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:13.373203 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:13.373203 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:13.373203 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:13.373203 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:13.373203 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:51:13.373203 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:51:13.373203 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:51:13.374558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:51:13.374558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:13.374558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:13.374558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:13.374558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:13.374558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:13.374558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:13.374558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:13.374558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:13.374558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:13.374558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:13.374558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:13.374558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:13.374558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:13.374558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:13.374558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:13.374558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:13.376082 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:51:13.376082 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:51:13.376082 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:51:13.376082 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:51:13.376082 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:51:13.376082 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:51:13.376082 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:51:13.376082 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:51:13.376082 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 542/1101] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:51:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:51:13.380305 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54926c5f-99ee-4236-b4d4-5eb23049b2fb None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:13.392156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 560/1102] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:51:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 19:51:13.393965 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb5f70-bd53-4d86-9fd2-c18de4ce5850 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:13.485000 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:13.492181 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54926c5f-99ee-4236-b4d4-5eb23049b2fb None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:13.512567 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb5f70-bd53-4d86-9fd2-c18de4ce5850 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:13.529951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f9597-b02b-479d-a97d-a2802ac0c8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:13.530572 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f9597-b02b-479d-a97d-a2802ac0c8e0 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:13.530848 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f9597-b02b-479d-a97d-a2802ac0c8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:13.531230 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f9597-b02b-479d-a97d-a2802ac0c8e0 None admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:13.531867 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f9597-b02b-479d-a97d-a2802ac0c8e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:13.532174 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f9597-b02b-479d-a97d-a2802ac0c8e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:13.533678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f9597-b02b-479d-a97d-a2802ac0c8e0 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:13.547691 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 543/1103] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:51:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:51:13.551805 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee188e-5c4b-46ae-a2bf-ec865187d7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:13.552284 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee188e-5c4b-46ae-a2bf-ec865187d7cb None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:13.552561 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee188e-5c4b-46ae-a2bf-ec865187d7cb None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:13.552859 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee188e-5c4b-46ae-a2bf-ec865187d7cb None admin] PATH_INFO: `/v3/projects/b2b9353feedb4cbaada0bc357a07613b` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:13.554086 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ee188e-5c4b-46ae-a2bf-ec865187d7cb None admin] RBAC: Authorizing `identity:get_project(project_id=b2b9353feedb4cbaada0bc357a07613b)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:13.554720 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f9597-b02b-479d-a97d-a2802ac0c8e0 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:13.555774 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ee188e-5c4b-46ae-a2bf-ec865187d7cb None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:13.563852 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 561/1104] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:51:13 2022] GET /identity/v3/projects/b2b9353feedb4cbaada0bc357a07613b => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:51:13.564766 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee188e-5c4b-46ae-a2bf-ec865187d7cb None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:13.592779 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:13.677520 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f9597-b02b-479d-a97d-a2802ac0c8e0 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:13.690654 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee188e-5c4b-46ae-a2bf-ec865187d7cb None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:13.716774 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc104c32-6bcf-41fb-9cdb-dbd9ec99d8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:13.717410 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc104c32-6bcf-41fb-9cdb-dbd9ec99d8b7 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:13.717691 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc104c32-6bcf-41fb-9cdb-dbd9ec99d8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:13.717964 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc104c32-6bcf-41fb-9cdb-dbd9ec99d8b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:13.718832 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc104c32-6bcf-41fb-9cdb-dbd9ec99d8b7 None admin] RBAC: Adding query filter params (role.id=0a40666ef2c140d1951dea50f3127507, scope.project.id=80379e2104644e98a2e85d4e4af1b365, user.id=95e9f085e4f04d118373b8abff27543a) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:13.719446 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc104c32-6bcf-41fb-9cdb-dbd9ec99d8b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:13.721187 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc104c32-6bcf-41fb-9cdb-dbd9ec99d8b7 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:13.729361 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 544/1105] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:51:13 2022] GET /identity/v3/role_assignments?user.id=95e9f085e4f04d118373b8abff27543a&scope.project.id=80379e2104644e98a2e85d4e4af1b365&role.id=0a40666ef2c140d1951dea50f3127507 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:51:13.731291 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73b342-926e-4653-8d0f-38cd8336ed5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:13.731902 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73b342-926e-4653-8d0f-38cd8336ed5f None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:13.732184 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73b342-926e-4653-8d0f-38cd8336ed5f None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:13.732484 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73b342-926e-4653-8d0f-38cd8336ed5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:13.733301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73b342-926e-4653-8d0f-38cd8336ed5f None admin] RBAC: Adding query filter params (role.id=89919f784ad144529b07c8ab92ee7d48, scope.project.id=b2b9353feedb4cbaada0bc357a07613b, user.id=5bba4c33cbe34e95906c7486676515d7) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:13.737813 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73b342-926e-4653-8d0f-38cd8336ed5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:13.737813 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73b342-926e-4653-8d0f-38cd8336ed5f None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:13.745549 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 562/1106] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:51:13 2022] GET /identity/v3/role_assignments?user.id=5bba4c33cbe34e95906c7486676515d7&scope.project.id=b2b9353feedb4cbaada0bc357a07613b&role.id=89919f784ad144529b07c8ab92ee7d48 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:51:15.889588 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab200fc-af07-4f02-aef7-8172d49df954 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:15.890133 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab200fc-af07-4f02-aef7-8172d49df954 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:15.890399 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab200fc-af07-4f02-aef7-8172d49df954 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:15.891799 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 545/1107] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:51:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:15.897907 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf005f5-d550-4594-ba5f-4e7cc035a963 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:15.899155 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf005f5-d550-4594-ba5f-4e7cc035a963 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:15.899155 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf005f5-d550-4594-ba5f-4e7cc035a963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:15.919499 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-cdf005f5-d550-4594-ba5f-4e7cc035a963 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:15.920741 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-cdf005f5-d550-4594-ba5f-4e7cc035a963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:15.923715 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:15.991800 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 563/1108] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:51:16.016987 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab200fc-af07-4f02-aef7-8172d49df954 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:16.126014 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:16.133619 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab200fc-af07-4f02-aef7-8172d49df954 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:16.168707 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e8f615-15bf-41d0-aee4-3e6821d79657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:16.169345 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8f615-15bf-41d0-aee4-3e6821d79657 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:16.169622 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8f615-15bf-41d0-aee4-3e6821d79657 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:16.169897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8f615-15bf-41d0-aee4-3e6821d79657 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:16.171027 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e8f615-15bf-41d0-aee4-3e6821d79657 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:16.172445 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e8f615-15bf-41d0-aee4-3e6821d79657 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:16.174065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 546/1109] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:51:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:16.179828 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf005f5-d550-4594-ba5f-4e7cc035a963 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:16.278679 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf005f5-d550-4594-ba5f-4e7cc035a963 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:16.311775 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6fc6f-30a5-4851-af42-57e08c5bf804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:16.312377 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6fc6f-30a5-4851-af42-57e08c5bf804 None admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:16.312682 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6fc6f-30a5-4851-af42-57e08c5bf804 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:16.312957 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6fc6f-30a5-4851-af42-57e08c5bf804 None admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:16.313643 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be6fc6f-30a5-4851-af42-57e08c5bf804 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:16.315065 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be6fc6f-30a5-4851-af42-57e08c5bf804 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:16.326936 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:16.338225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 564/1110] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Aug 5 19:51:16 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:51:18.413625 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae59be-9780-4533-bb22-471411d1a016 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:18.414643 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae59be-9780-4533-bb22-471411d1a016 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:18.414643 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae59be-9780-4533-bb22-471411d1a016 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:18.415973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 547/1111] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:18.421111 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9551eb-3e7c-41a8-9279-20113eb3e9b3 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:18.421528 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9551eb-3e7c-41a8-9279-20113eb3e9b3 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:18.421665 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9551eb-3e7c-41a8-9279-20113eb3e9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:18.441706 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-7e9551eb-3e7c-41a8-9279-20113eb3e9b3 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:18.442892 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-7e9551eb-3e7c-41a8-9279-20113eb3e9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:18.510057 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 565/1112] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:51:18.533936 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae59be-9780-4533-bb22-471411d1a016 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:18.633623 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae59be-9780-4533-bb22-471411d1a016 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:18.667978 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6678ce8b-f2e3-4405-8b19-060621b9a62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:18.668531 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6678ce8b-f2e3-4405-8b19-060621b9a62a None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:18.668853 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6678ce8b-f2e3-4405-8b19-060621b9a62a None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:18.669128 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6678ce8b-f2e3-4405-8b19-060621b9a62a None admin] PATH_INFO: `/v3/roles/0eafd8a7ba2d47faaf5aecbb3360143b` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:18.670558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6678ce8b-f2e3-4405-8b19-060621b9a62a None admin] RBAC: Authorizing `identity:get_role(role_id=0eafd8a7ba2d47faaf5aecbb3360143b)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:18.671908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6678ce8b-f2e3-4405-8b19-060621b9a62a None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:18.673281 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 548/1113] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:51:18 2022] GET /identity/v3/roles/0eafd8a7ba2d47faaf5aecbb3360143b => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:18.679261 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9551eb-3e7c-41a8-9279-20113eb3e9b3 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:18.776447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9551eb-3e7c-41a8-9279-20113eb3e9b3 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:18.801996 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:18.811247 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe59b15-8198-4cdc-9f69-3e94ecae4e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:18.811877 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe59b15-8198-4cdc-9f69-3e94ecae4e23 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:18.812155 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe59b15-8198-4cdc-9f69-3e94ecae4e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:18.812447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe59b15-8198-4cdc-9f69-3e94ecae4e23 None admin] PATH_INFO: `/v3/users/105d89698b854d9c9ef374bc18e4db6b` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:18.826078 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe59b15-8198-4cdc-9f69-3e94ecae4e23 None admin] RBAC: Authorizing `identity:get_user(user_id=105d89698b854d9c9ef374bc18e4db6b)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:18.827501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe59b15-8198-4cdc-9f69-3e94ecae4e23 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:18.829144 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 566/1114] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:51:18 2022] GET /identity/v3/users/105d89698b854d9c9ef374bc18e4db6b => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:18.835051 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6678ce8b-f2e3-4405-8b19-060621b9a62a None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:18.866780 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:18.933771 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6678ce8b-f2e3-4405-8b19-060621b9a62a None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:18.966312 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a780566-a3ac-4340-b5ae-9dbff57308ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:18.966906 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a780566-a3ac-4340-b5ae-9dbff57308ee None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:18.967181 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a780566-a3ac-4340-b5ae-9dbff57308ee None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:18.967448 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a780566-a3ac-4340-b5ae-9dbff57308ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:19.032672 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a780566-a3ac-4340-b5ae-9dbff57308ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:19.034088 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a780566-a3ac-4340-b5ae-9dbff57308ee None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:19.062126 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:19.132464 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 549/1115] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:51:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:51:19.138589 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe59b15-8198-4cdc-9f69-3e94ecae4e23 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:19.235696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe59b15-8198-4cdc-9f69-3e94ecae4e23 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:19.269132 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d8defd-a55a-462b-b647-21f6c0510fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:19.269748 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8defd-a55a-462b-b647-21f6c0510fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:19.270026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8defd-a55a-462b-b647-21f6c0510fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:19.270296 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8defd-a55a-462b-b647-21f6c0510fa7 None admin] PATH_INFO: `/v3/projects/b2b9353feedb4cbaada0bc357a07613b` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:19.271415 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d8defd-a55a-462b-b647-21f6c0510fa7 None admin] RBAC: Authorizing `identity:get_project(project_id=b2b9353feedb4cbaada0bc357a07613b)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:19.273125 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d8defd-a55a-462b-b647-21f6c0510fa7 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:19.274929 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 567/1116] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:51:19 2022] GET /identity/v3/projects/b2b9353feedb4cbaada0bc357a07613b => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:19.281242 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a780566-a3ac-4340-b5ae-9dbff57308ee None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:19.379511 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a780566-a3ac-4340-b5ae-9dbff57308ee None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:19.408715 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:19.417321 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1171d8-aab7-484c-96a7-968d4140356e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:19.417952 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1171d8-aab7-484c-96a7-968d4140356e None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:19.418203 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1171d8-aab7-484c-96a7-968d4140356e None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:19.418473 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1171d8-aab7-484c-96a7-968d4140356e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:19.419262 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1171d8-aab7-484c-96a7-968d4140356e None admin] RBAC: Adding query filter params (role.id=0eafd8a7ba2d47faaf5aecbb3360143b, scope.project.id=b2b9353feedb4cbaada0bc357a07613b, user.id=105d89698b854d9c9ef374bc18e4db6b) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:19.419570 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1171d8-aab7-484c-96a7-968d4140356e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:19.421017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1171d8-aab7-484c-96a7-968d4140356e None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:19.426694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 550/1117] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:51:19 2022] GET /identity/v3/role_assignments?user.id=105d89698b854d9c9ef374bc18e4db6b&scope.project.id=b2b9353feedb4cbaada0bc357a07613b&role.id=0eafd8a7ba2d47faaf5aecbb3360143b => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:21.401219 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73032a-1988-408e-b3e7-54e5a4ec0925 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:21.401725 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73032a-1988-408e-b3e7-54e5a4ec0925 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:21.401725 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73032a-1988-408e-b3e7-54e5a4ec0925 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:21.403016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 568/1118] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:21.408123 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b606b-e452-417b-bbd8-93a3066b1cd6 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:21.408502 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b606b-e452-417b-bbd8-93a3066b1cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:21.408763 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b606b-e452-417b-bbd8-93a3066b1cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:21.427996 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-db3b606b-e452-417b-bbd8-93a3066b1cd6 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:21.429209 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-db3b606b-e452-417b-bbd8-93a3066b1cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:21.497557 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 551/1119] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:51:21.519537 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73032a-1988-408e-b3e7-54e5a4ec0925 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:21.576343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:21.626743 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73032a-1988-408e-b3e7-54e5a4ec0925 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:21.661171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532809e-0460-49d4-9fdc-fd3fee926651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:21.661728 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532809e-0460-49d4-9fdc-fd3fee926651 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:21.662006 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532809e-0460-49d4-9fdc-fd3fee926651 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:21.662279 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532809e-0460-49d4-9fdc-fd3fee926651 None admin] PATH_INFO: `/v3/roles/0eafd8a7ba2d47faaf5aecbb3360143b` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:21.663694 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532809e-0460-49d4-9fdc-fd3fee926651 None admin] RBAC: Authorizing `identity:get_role(role_id=0eafd8a7ba2d47faaf5aecbb3360143b)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:21.666011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532809e-0460-49d4-9fdc-fd3fee926651 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:21.667343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 569/1120] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:51:21 2022] GET /identity/v3/roles/0eafd8a7ba2d47faaf5aecbb3360143b => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:21.673203 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b606b-e452-417b-bbd8-93a3066b1cd6 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:21.769797 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b606b-e452-417b-bbd8-93a3066b1cd6 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:21.795765 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:21.804929 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e8fc9-aa42-46ec-9372-db7c4463a459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:21.805556 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e8fc9-aa42-46ec-9372-db7c4463a459 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:21.805834 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e8fc9-aa42-46ec-9372-db7c4463a459 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:21.806142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e8fc9-aa42-46ec-9372-db7c4463a459 None admin] PATH_INFO: `/v3/users/105d89698b854d9c9ef374bc18e4db6b` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:21.807424 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6e8fc9-aa42-46ec-9372-db7c4463a459 None admin] RBAC: Authorizing `identity:get_user(user_id=105d89698b854d9c9ef374bc18e4db6b)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:21.808901 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6e8fc9-aa42-46ec-9372-db7c4463a459 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:21.810556 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 552/1121] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:51:21 2022] GET /identity/v3/users/105d89698b854d9c9ef374bc18e4db6b => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:21.816641 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532809e-0460-49d4-9fdc-fd3fee926651 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:21.915215 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532809e-0460-49d4-9fdc-fd3fee926651 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:21.934913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:21.949379 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5a2bf-ec3a-4e88-afea-fd4037f5dd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:21.950010 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5a2bf-ec3a-4e88-afea-fd4037f5dd94 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:21.950291 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5a2bf-ec3a-4e88-afea-fd4037f5dd94 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:21.950567 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5a2bf-ec3a-4e88-afea-fd4037f5dd94 None admin] PATH_INFO: `/v3/projects/b2b9353feedb4cbaada0bc357a07613b` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:21.951715 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d5a2bf-ec3a-4e88-afea-fd4037f5dd94 None admin] RBAC: Authorizing `identity:get_project(project_id=b2b9353feedb4cbaada0bc357a07613b)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:21.953171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d5a2bf-ec3a-4e88-afea-fd4037f5dd94 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:21.954836 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 570/1122] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:51:21 2022] GET /identity/v3/projects/b2b9353feedb4cbaada0bc357a07613b => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:21.963017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e8fc9-aa42-46ec-9372-db7c4463a459 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:22.067984 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e8fc9-aa42-46ec-9372-db7c4463a459 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:22.122325 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e36d0-a539-4650-8d15-779f3d2c1f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:22.123485 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e36d0-a539-4650-8d15-779f3d2c1f90 None admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:22.123968 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e36d0-a539-4650-8d15-779f3d2c1f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:22.124463 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e36d0-a539-4650-8d15-779f3d2c1f90 None admin] PATH_INFO: `/v3/projects/b2b9353feedb4cbaada0bc357a07613b/users/105d89698b854d9c9ef374bc18e4db6b/roles/0eafd8a7ba2d47faaf5aecbb3360143b` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:22.128403 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75e36d0-a539-4650-8d15-779f3d2c1f90 None admin] RBAC: Authorizing `identity:create_grant(project_id=b2b9353feedb4cbaada0bc357a07613b, user_id=105d89698b854d9c9ef374bc18e4db6b, role_id=0eafd8a7ba2d47faaf5aecbb3360143b)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:22.129904 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75e36d0-a539-4650-8d15-779f3d2c1f90 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:22.151652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 553/1123] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Aug 5 19:51:21 2022] PUT /identity/v3/projects/b2b9353feedb4cbaada0bc357a07613b/users/105d89698b854d9c9ef374bc18e4db6b/roles/0eafd8a7ba2d47faaf5aecbb3360143b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:24.175147 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9549bba-79f0-4a10-ad9b-f24de7969bf4 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:24.175621 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9549bba-79f0-4a10-ad9b-f24de7969bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:24.175712 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9549bba-79f0-4a10-ad9b-f24de7969bf4 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:24.177086 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 571/1124] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:24.182900 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c97c60-7995-4349-ba6b-eb759ec37b74 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:24.183336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c97c60-7995-4349-ba6b-eb759ec37b74 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:24.183426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c97c60-7995-4349-ba6b-eb759ec37b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:24.203015 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-b6c97c60-7995-4349-ba6b-eb759ec37b74 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:24.204201 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-b6c97c60-7995-4349-ba6b-eb759ec37b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:24.255975 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:24.276783 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 554/1125] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:51:24.299898 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9549bba-79f0-4a10-ad9b-f24de7969bf4 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:24.402904 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9549bba-79f0-4a10-ad9b-f24de7969bf4 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:24.447872 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f228734-e353-4c33-99bf-9021e511841a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:24.448512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f228734-e353-4c33-99bf-9021e511841a None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:24.448855 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f228734-e353-4c33-99bf-9021e511841a None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:24.449102 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f228734-e353-4c33-99bf-9021e511841a None admin] PATH_INFO: `/v3/roles/0eafd8a7ba2d47faaf5aecbb3360143b` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:24.450654 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f228734-e353-4c33-99bf-9021e511841a None admin] RBAC: Authorizing `identity:get_role(role_id=0eafd8a7ba2d47faaf5aecbb3360143b)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:24.452031 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f228734-e353-4c33-99bf-9021e511841a None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:24.453442 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 572/1126] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:51:24 2022] GET /identity/v3/roles/0eafd8a7ba2d47faaf5aecbb3360143b => generated 223 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:24.459311 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c97c60-7995-4349-ba6b-eb759ec37b74 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:24.558165 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c97c60-7995-4349-ba6b-eb759ec37b74 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:24.592729 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf21e5-0982-4db1-ad88-711ddaf05f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:24.593378 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf21e5-0982-4db1-ad88-711ddaf05f05 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:24.593658 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf21e5-0982-4db1-ad88-711ddaf05f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:24.593932 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf21e5-0982-4db1-ad88-711ddaf05f05 None admin] PATH_INFO: `/v3/users/105d89698b854d9c9ef374bc18e4db6b` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:24.595075 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cf21e5-0982-4db1-ad88-711ddaf05f05 None admin] RBAC: Authorizing `identity:get_user(user_id=105d89698b854d9c9ef374bc18e4db6b)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:24.596494 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cf21e5-0982-4db1-ad88-711ddaf05f05 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:24.598113 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 555/1127] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:51:24 2022] GET /identity/v3/users/105d89698b854d9c9ef374bc18e4db6b => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:24.604238 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f228734-e353-4c33-99bf-9021e511841a None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:24.660489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:24.703951 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f228734-e353-4c33-99bf-9021e511841a None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:24.740407 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce137df-77c9-4e8b-b0a1-ab9c4ceec495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:24.741042 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce137df-77c9-4e8b-b0a1-ab9c4ceec495 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:24.741324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce137df-77c9-4e8b-b0a1-ab9c4ceec495 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:24.741598 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce137df-77c9-4e8b-b0a1-ab9c4ceec495 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:24.812389 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce137df-77c9-4e8b-b0a1-ab9c4ceec495 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:24.813877 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce137df-77c9-4e8b-b0a1-ab9c4ceec495 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:24.867367 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:24.910336 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 573/1128] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:51:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:51:24.917241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf21e5-0982-4db1-ad88-711ddaf05f05 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:24.969264 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:25.016811 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf21e5-0982-4db1-ad88-711ddaf05f05 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:25.050177 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca920ca-252e-4900-a0f5-0064f7456106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:25.050808 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca920ca-252e-4900-a0f5-0064f7456106 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:25.051091 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca920ca-252e-4900-a0f5-0064f7456106 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:25.051364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca920ca-252e-4900-a0f5-0064f7456106 None admin] PATH_INFO: `/v3/projects/b2b9353feedb4cbaada0bc357a07613b` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:25.052620 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca920ca-252e-4900-a0f5-0064f7456106 None admin] RBAC: Authorizing `identity:get_project(project_id=b2b9353feedb4cbaada0bc357a07613b)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:25.054043 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca920ca-252e-4900-a0f5-0064f7456106 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:25.056301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 556/1129] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:51:24 2022] GET /identity/v3/projects/b2b9353feedb4cbaada0bc357a07613b => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:25.062453 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce137df-77c9-4e8b-b0a1-ab9c4ceec495 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:25.165410 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce137df-77c9-4e8b-b0a1-ab9c4ceec495 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:25.199150 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd56742-f832-4475-b1ab-a2ad446f6372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:25.199703 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd56742-f832-4475-b1ab-a2ad446f6372 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:25.199957 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd56742-f832-4475-b1ab-a2ad446f6372 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:25.200231 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd56742-f832-4475-b1ab-a2ad446f6372 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:25.201043 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd56742-f832-4475-b1ab-a2ad446f6372 None admin] RBAC: Adding query filter params (role.id=0eafd8a7ba2d47faaf5aecbb3360143b, scope.project.id=b2b9353feedb4cbaada0bc357a07613b, user.id=105d89698b854d9c9ef374bc18e4db6b) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:25.201353 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd56742-f832-4475-b1ab-a2ad446f6372 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:25.202773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd56742-f832-4475-b1ab-a2ad446f6372 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:25.208873 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 574/1130] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:51:25 2022] GET /identity/v3/role_assignments?user.id=105d89698b854d9c9ef374bc18e4db6b&scope.project.id=b2b9353feedb4cbaada0bc357a07613b&role.id=0eafd8a7ba2d47faaf5aecbb3360143b => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:27.222520 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acde247-f062-4fdd-86e1-368c18f00533 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:27.223071 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acde247-f062-4fdd-86e1-368c18f00533 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:27.223071 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acde247-f062-4fdd-86e1-368c18f00533 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:27.224393 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 557/1131] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:51:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:27.230304 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164c102-7e73-4005-b7b5-0cd4503b49db None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:27.230710 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164c102-7e73-4005-b7b5-0cd4503b49db None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:27.230865 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164c102-7e73-4005-b7b5-0cd4503b49db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:27.249645 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-0164c102-7e73-4005-b7b5-0cd4503b49db None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:27.250827 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-0164c102-7e73-4005-b7b5-0cd4503b49db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:27.265099 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:27.315900 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 575/1132] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:51:27.340170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acde247-f062-4fdd-86e1-368c18f00533 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:27.440238 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acde247-f062-4fdd-86e1-368c18f00533 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:27.460214 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:27.474727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee05b5d-d66d-4e21-af3c-017dabd09f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:27.475347 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee05b5d-d66d-4e21-af3c-017dabd09f77 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:27.475621 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee05b5d-d66d-4e21-af3c-017dabd09f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:27.475912 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee05b5d-d66d-4e21-af3c-017dabd09f77 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:27.477156 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee05b5d-d66d-4e21-af3c-017dabd09f77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:27.478562 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee05b5d-d66d-4e21-af3c-017dabd09f77 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:27.480124 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 558/1133] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:51:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:27.486037 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0164c102-7e73-4005-b7b5-0cd4503b49db None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:27.584486 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0164c102-7e73-4005-b7b5-0cd4503b49db None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:27.618930 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf527d9b-c219-40e8-8edd-ac5118ec32fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:27.619540 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf527d9b-c219-40e8-8edd-ac5118ec32fa None admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:27.619846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf527d9b-c219-40e8-8edd-ac5118ec32fa None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:27.620119 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf527d9b-c219-40e8-8edd-ac5118ec32fa None admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:27.620834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf527d9b-c219-40e8-8edd-ac5118ec32fa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:27.622256 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf527d9b-c219-40e8-8edd-ac5118ec32fa None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:27.649767 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 576/1134] 10.222.0.39 () {60 vars in 1141 bytes} [Fri Aug 5 19:51:27 2022] POST /identity/v3/projects => generated 311 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:51:29.723003 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf614a6d-6c42-4868-a84d-ce2d8b299e56 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:29.723537 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf614a6d-6c42-4868-a84d-ce2d8b299e56 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:29.723537 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf614a6d-6c42-4868-a84d-ce2d8b299e56 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:29.724933 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 559/1135] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:29.730811 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc36a5-afac-44a0-88fd-93a4988a574e None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:29.731233 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc36a5-afac-44a0-88fd-93a4988a574e None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:29.731374 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc36a5-afac-44a0-88fd-93a4988a574e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:29.737716 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:29.751710 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-77cc36a5-afac-44a0-88fd-93a4988a574e None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:29.752938 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-77cc36a5-afac-44a0-88fd-93a4988a574e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:29.819269 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 577/1136] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:51:29.843806 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf614a6d-6c42-4868-a84d-ce2d8b299e56 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:29.944828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf614a6d-6c42-4868-a84d-ce2d8b299e56 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:29.979214 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8019e85-779e-43c8-a34d-3026652b9be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:29.979769 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8019e85-779e-43c8-a34d-3026652b9be6 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:29.980051 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8019e85-779e-43c8-a34d-3026652b9be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:29.980329 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8019e85-779e-43c8-a34d-3026652b9be6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:29.981508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8019e85-779e-43c8-a34d-3026652b9be6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:29.982924 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8019e85-779e-43c8-a34d-3026652b9be6 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:29.984495 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 560/1137] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:51:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:29.990448 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc36a5-afac-44a0-88fd-93a4988a574e None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:30.085418 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc36a5-afac-44a0-88fd-93a4988a574e None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:30.114196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:30.120353 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768bbec3-2d96-41ea-85b4-4ceb49464df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:30.120996 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bbec3-2d96-41ea-85b4-4ceb49464df0 None admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:30.121275 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bbec3-2d96-41ea-85b4-4ceb49464df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:30.121543 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bbec3-2d96-41ea-85b4-4ceb49464df0 None admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:30.122215 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768bbec3-2d96-41ea-85b4-4ceb49464df0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:30.123633 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768bbec3-2d96-41ea-85b4-4ceb49464df0 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:30.144897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 578/1138] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Aug 5 19:51:29 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:51:32.205019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfa587-1813-4abe-92a9-dfcd7b41cfbe None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:32.205560 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfa587-1813-4abe-92a9-dfcd7b41cfbe None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:32.205560 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfa587-1813-4abe-92a9-dfcd7b41cfbe None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:32.206916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 561/1139] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:32.212145 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725b580-f586-4b8f-8d06-1a69c5c98caf None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:32.212551 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725b580-f586-4b8f-8d06-1a69c5c98caf None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:32.212720 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725b580-f586-4b8f-8d06-1a69c5c98caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:32.230722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-4725b580-f586-4b8f-8d06-1a69c5c98caf None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:32.231890 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-4725b580-f586-4b8f-8d06-1a69c5c98caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:32.312295 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 579/1140] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:51:32.336219 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dfa587-1813-4abe-92a9-dfcd7b41cfbe None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:32.396429 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:32.440789 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dfa587-1813-4abe-92a9-dfcd7b41cfbe None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:32.474494 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd565c51-9ff1-4256-a1dd-a9baef9a75a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:32.475112 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd565c51-9ff1-4256-a1dd-a9baef9a75a0 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:32.475395 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd565c51-9ff1-4256-a1dd-a9baef9a75a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:32.475747 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd565c51-9ff1-4256-a1dd-a9baef9a75a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:32.486160 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd565c51-9ff1-4256-a1dd-a9baef9a75a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:51:32.491165 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-dd565c51-9ff1-4256-a1dd-a9baef9a75a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:51:32.491165 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:32.491165 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:32.491165 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:32.491165 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:32.491165 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:32.491165 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:32.491165 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:32.491165 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:32.491165 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:32.491165 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:32.491165 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:32.491165 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:51:32.491165 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:51:32.491165 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:51:32.491165 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:51:32.493480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:51:32.493480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:51:32.493480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:51:32.493480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:51:32.493480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:32.493480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:32.493480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:32.493480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:32.493480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:32.493480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:32.493480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:32.493480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:32.493480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:32.493480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:32.493480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:32.493480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:32.494454 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:32.494454 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:32.494454 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:32.494454 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:32.494454 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:51:32.494454 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:51:32.494454 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:51:32.494454 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:51:32.494454 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:51:32.494454 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:51:32.494454 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:51:32.494454 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:51:32.494454 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 562/1141] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:51:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:51:32.499005 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4725b580-f586-4b8f-8d06-1a69c5c98caf None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:32.611228 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4725b580-f586-4b8f-8d06-1a69c5c98caf None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:32.640433 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:32.660038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5a68a-8222-4503-a33c-9d4cb4931982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:32.660600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5a68a-8222-4503-a33c-9d4cb4931982 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:32.660888 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5a68a-8222-4503-a33c-9d4cb4931982 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:32.661158 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5a68a-8222-4503-a33c-9d4cb4931982 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:32.661861 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc5a68a-8222-4503-a33c-9d4cb4931982 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:32.662164 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc5a68a-8222-4503-a33c-9d4cb4931982 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:32.663643 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc5a68a-8222-4503-a33c-9d4cb4931982 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:32.671459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 580/1142] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:51:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:32.677457 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd565c51-9ff1-4256-a1dd-a9baef9a75a0 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:32.776078 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd565c51-9ff1-4256-a1dd-a9baef9a75a0 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:32.792160 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:32.810369 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22cb1b-3195-4898-8e75-84440220fd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:32.810989 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22cb1b-3195-4898-8e75-84440220fd24 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:32.811265 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22cb1b-3195-4898-8e75-84440220fd24 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:32.811533 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22cb1b-3195-4898-8e75-84440220fd24 None admin] PATH_INFO: `/v3/users/20d4920e70d74c6ca0fa3e1181ed0edf` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:32.825593 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c22cb1b-3195-4898-8e75-84440220fd24 None admin] RBAC: Authorizing `identity:get_user(user_id=20d4920e70d74c6ca0fa3e1181ed0edf)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:32.827011 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c22cb1b-3195-4898-8e75-84440220fd24 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:32.828634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 563/1143] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:51:32 2022] GET /identity/v3/users/20d4920e70d74c6ca0fa3e1181ed0edf => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:32.834454 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5a68a-8222-4503-a33c-9d4cb4931982 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:32.940276 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5a68a-8222-4503-a33c-9d4cb4931982 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:32.973454 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b95336-90c6-4a91-b96e-f8395563d3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:32.974122 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b95336-90c6-4a91-b96e-f8395563d3bc None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:32.974465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b95336-90c6-4a91-b96e-f8395563d3bc None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:32.974802 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b95336-90c6-4a91-b96e-f8395563d3bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:33.023959 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:33.044387 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b95336-90c6-4a91-b96e-f8395563d3bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:33.045963 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b95336-90c6-4a91-b96e-f8395563d3bc None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:33.143928 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 581/1144] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:51:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:51:33.150727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22cb1b-3195-4898-8e75-84440220fd24 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:33.248116 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22cb1b-3195-4898-8e75-84440220fd24 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:33.282223 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86576e-e80d-474a-ba87-3cae16e4d925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:33.282856 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86576e-e80d-474a-ba87-3cae16e4d925 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:33.283141 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86576e-e80d-474a-ba87-3cae16e4d925 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:33.283418 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86576e-e80d-474a-ba87-3cae16e4d925 None admin] PATH_INFO: `/v3/projects/7266218fba7042fca5543bd3c34f3420` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:33.284561 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e86576e-e80d-474a-ba87-3cae16e4d925 None admin] RBAC: Authorizing `identity:get_project(project_id=7266218fba7042fca5543bd3c34f3420)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:33.286017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e86576e-e80d-474a-ba87-3cae16e4d925 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:33.287662 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 564/1145] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:51:33 2022] GET /identity/v3/projects/7266218fba7042fca5543bd3c34f3420 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:33.294323 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b95336-90c6-4a91-b96e-f8395563d3bc None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:33.372651 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:33.391219 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b95336-90c6-4a91-b96e-f8395563d3bc None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:33.424967 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971dded7-7759-4b99-bb85-69d72998e136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:33.425563 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dded7-7759-4b99-bb85-69d72998e136 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:33.425838 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dded7-7759-4b99-bb85-69d72998e136 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:33.426108 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dded7-7759-4b99-bb85-69d72998e136 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:33.426887 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971dded7-7759-4b99-bb85-69d72998e136 None admin] RBAC: Adding query filter params (role.id=89919f784ad144529b07c8ab92ee7d48, scope.project.id=7266218fba7042fca5543bd3c34f3420, user.id=20d4920e70d74c6ca0fa3e1181ed0edf) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:33.427191 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971dded7-7759-4b99-bb85-69d72998e136 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:33.428737 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971dded7-7759-4b99-bb85-69d72998e136 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:33.435429 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 582/1146] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:51:33 2022] GET /identity/v3/role_assignments?user.id=20d4920e70d74c6ca0fa3e1181ed0edf&scope.project.id=7266218fba7042fca5543bd3c34f3420&role.id=89919f784ad144529b07c8ab92ee7d48 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:35.425017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca240dc-1fc4-4d1a-899c-ac418e975609 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:35.426848 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca240dc-1fc4-4d1a-899c-ac418e975609 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:35.426848 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca240dc-1fc4-4d1a-899c-ac418e975609 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:35.427476 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 565/1147] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:51:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:35.432788 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15083377-e809-4a47-9559-e7d3ff022e5e None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:35.433205 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15083377-e809-4a47-9559-e7d3ff022e5e None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:35.433337 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15083377-e809-4a47-9559-e7d3ff022e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:35.451095 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-15083377-e809-4a47-9559-e7d3ff022e5e None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:35.452268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-15083377-e809-4a47-9559-e7d3ff022e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:35.521172 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 583/1148] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:51:35.544133 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca240dc-1fc4-4d1a-899c-ac418e975609 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:35.589158 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:35.646321 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca240dc-1fc4-4d1a-899c-ac418e975609 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:35.680501 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e4258-0e91-4c68-ac20-5f4b96adb386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:35.681147 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e4258-0e91-4c68-ac20-5f4b96adb386 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:35.681425 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e4258-0e91-4c68-ac20-5f4b96adb386 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:35.681696 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e4258-0e91-4c68-ac20-5f4b96adb386 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:35.691964 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8e4258-0e91-4c68-ac20-5f4b96adb386 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:51:35.697404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-6d8e4258-0e91-4c68-ac20-5f4b96adb386 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:51:35.697404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:35.697404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:35.697404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:35.697404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:35.697404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:35.697404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:35.697404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:35.697404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:35.697404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:35.697404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:35.697404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:35.697404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:51:35.697404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:51:35.697404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:51:35.697404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:51:35.698521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:51:35.698521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:51:35.698521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:51:35.698521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:51:35.698521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:35.698521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:35.698521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:35.698521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:35.698521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:35.698521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:35.698521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:35.698521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:35.698521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:35.698521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:35.698521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:35.698521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:35.704397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:35.704397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:35.704397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:35.704397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:35.704397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:51:35.704397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:51:35.704397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:51:35.704397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:51:35.704397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:51:35.704397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:51:35.704397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:51:35.704397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:51:35.704397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 566/1149] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:51:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:51:35.705301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15083377-e809-4a47-9559-e7d3ff022e5e None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:35.779586 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:35.804847 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15083377-e809-4a47-9559-e7d3ff022e5e None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:35.838594 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed66bc9b-8df3-429a-8d1b-6ec4bc4fcb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:35.839203 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66bc9b-8df3-429a-8d1b-6ec4bc4fcb57 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:35.839483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66bc9b-8df3-429a-8d1b-6ec4bc4fcb57 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:35.839762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66bc9b-8df3-429a-8d1b-6ec4bc4fcb57 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:35.840467 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed66bc9b-8df3-429a-8d1b-6ec4bc4fcb57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:35.840815 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed66bc9b-8df3-429a-8d1b-6ec4bc4fcb57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:35.842161 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed66bc9b-8df3-429a-8d1b-6ec4bc4fcb57 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:35.850096 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 584/1150] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:51:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:35.856139 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e4258-0e91-4c68-ac20-5f4b96adb386 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:35.955607 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e4258-0e91-4c68-ac20-5f4b96adb386 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:35.957073 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:35.990180 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481ff33-5e19-4762-9728-57f57e88f477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:35.990812 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481ff33-5e19-4762-9728-57f57e88f477 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:35.991750 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481ff33-5e19-4762-9728-57f57e88f477 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:35.992000 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481ff33-5e19-4762-9728-57f57e88f477 None admin] PATH_INFO: `/v3/users/20d4920e70d74c6ca0fa3e1181ed0edf` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:35.993187 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3481ff33-5e19-4762-9728-57f57e88f477 None admin] RBAC: Authorizing `identity:get_user(user_id=20d4920e70d74c6ca0fa3e1181ed0edf)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:35.994602 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3481ff33-5e19-4762-9728-57f57e88f477 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:35.996186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 567/1151] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:51:35 2022] GET /identity/v3/users/20d4920e70d74c6ca0fa3e1181ed0edf => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:36.002366 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed66bc9b-8df3-429a-8d1b-6ec4bc4fcb57 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:36.103856 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed66bc9b-8df3-429a-8d1b-6ec4bc4fcb57 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:36.137618 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355e9ba-490e-4aa5-854f-e45581e8d8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:36.138247 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355e9ba-490e-4aa5-854f-e45581e8d8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:36.138535 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355e9ba-490e-4aa5-854f-e45581e8d8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:36.138812 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355e9ba-490e-4aa5-854f-e45581e8d8c4 None admin] PATH_INFO: `/v3/projects/7266218fba7042fca5543bd3c34f3420` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:36.139982 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355e9ba-490e-4aa5-854f-e45581e8d8c4 None admin] RBAC: Authorizing `identity:get_project(project_id=7266218fba7042fca5543bd3c34f3420)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:36.141464 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355e9ba-490e-4aa5-854f-e45581e8d8c4 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:36.143165 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 585/1152] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:51:36 2022] GET /identity/v3/projects/7266218fba7042fca5543bd3c34f3420 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:36.149838 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481ff33-5e19-4762-9728-57f57e88f477 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:36.249725 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481ff33-5e19-4762-9728-57f57e88f477 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:36.283812 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff4226-9efa-43ae-a116-6bdf71af2e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:36.284463 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff4226-9efa-43ae-a116-6bdf71af2e55 None admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:36.284775 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff4226-9efa-43ae-a116-6bdf71af2e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:36.285054 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff4226-9efa-43ae-a116-6bdf71af2e55 None admin] PATH_INFO: `/v3/projects/7266218fba7042fca5543bd3c34f3420/users/20d4920e70d74c6ca0fa3e1181ed0edf/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:36.286939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff4226-9efa-43ae-a116-6bdf71af2e55 None admin] RBAC: Authorizing `identity:create_grant(project_id=7266218fba7042fca5543bd3c34f3420, user_id=20d4920e70d74c6ca0fa3e1181ed0edf, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:36.289461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff4226-9efa-43ae-a116-6bdf71af2e55 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:36.301833 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 568/1153] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Aug 5 19:51:36 2022] PUT /identity/v3/projects/7266218fba7042fca5543bd3c34f3420/users/20d4920e70d74c6ca0fa3e1181ed0edf/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:38.317035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf184a-5d69-4452-9ed5-860a649527c9 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:38.317602 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf184a-5d69-4452-9ed5-860a649527c9 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:38.317602 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf184a-5d69-4452-9ed5-860a649527c9 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:38.318934 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 586/1154] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:51:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:38.324158 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c296fd-afa8-499a-a6d2-bea2c832ec9b None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:38.324561 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c296fd-afa8-499a-a6d2-bea2c832ec9b None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:38.324751 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c296fd-afa8-499a-a6d2-bea2c832ec9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:38.343122 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-e7c296fd-afa8-499a-a6d2-bea2c832ec9b None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:38.344285 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-e7c296fd-afa8-499a-a6d2-bea2c832ec9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:38.367598 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:38.412068 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 569/1155] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:51:38.435020 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf184a-5d69-4452-9ed5-860a649527c9 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:38.438814 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:38.536643 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf184a-5d69-4452-9ed5-860a649527c9 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:38.570324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba84adc0-e5f0-47d4-99da-a49bbd74b948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:38.570944 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84adc0-e5f0-47d4-99da-a49bbd74b948 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:38.571222 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84adc0-e5f0-47d4-99da-a49bbd74b948 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:38.571493 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84adc0-e5f0-47d4-99da-a49bbd74b948 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:38.581698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba84adc0-e5f0-47d4-99da-a49bbd74b948 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:51:38.586934 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-ba84adc0-e5f0-47d4-99da-a49bbd74b948 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:51:38.586934 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:38.586934 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:38.586934 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:38.586934 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:38.586934 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:38.586934 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:38.586934 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:38.586934 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:38.586934 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:38.586934 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:38.586934 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:38.586934 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:51:38.586934 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:51:38.586934 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:51:38.586934 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:51:38.588073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:51:38.588073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:51:38.588073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:51:38.588073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:51:38.588073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:38.588073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:38.588073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:38.588073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:38.588073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:38.588073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:38.588073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:38.588073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:38.588073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:38.588073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:38.588073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:38.588073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:38.589181 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:38.589181 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:38.589181 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:38.589181 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:38.589181 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:51:38.589181 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:51:38.589181 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:51:38.589181 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:51:38.589181 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:51:38.589181 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:51:38.589181 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:51:38.589181 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:51:38.589181 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 587/1156] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:51:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:51:38.594165 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c296fd-afa8-499a-a6d2-bea2c832ec9b None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:38.705083 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c296fd-afa8-499a-a6d2-bea2c832ec9b None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:38.739010 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fabfa75-3b2f-4f25-8a00-402347f71b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:38.739610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fabfa75-3b2f-4f25-8a00-402347f71b44 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:38.739892 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fabfa75-3b2f-4f25-8a00-402347f71b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:38.740161 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fabfa75-3b2f-4f25-8a00-402347f71b44 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:38.741489 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fabfa75-3b2f-4f25-8a00-402347f71b44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:38.741788 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fabfa75-3b2f-4f25-8a00-402347f71b44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:38.743133 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fabfa75-3b2f-4f25-8a00-402347f71b44 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:38.750941 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 570/1157] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:51:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:38.756839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba84adc0-e5f0-47d4-99da-a49bbd74b948 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:38.816943 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:38.855143 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba84adc0-e5f0-47d4-99da-a49bbd74b948 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:38.891844 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb10152-2d0c-40b3-9377-bdace3662225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:38.892466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb10152-2d0c-40b3-9377-bdace3662225 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:38.892776 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb10152-2d0c-40b3-9377-bdace3662225 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:38.893052 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb10152-2d0c-40b3-9377-bdace3662225 None admin] PATH_INFO: `/v3/users/20d4920e70d74c6ca0fa3e1181ed0edf` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:38.894280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb10152-2d0c-40b3-9377-bdace3662225 None admin] RBAC: Authorizing `identity:get_user(user_id=20d4920e70d74c6ca0fa3e1181ed0edf)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:38.895703 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb10152-2d0c-40b3-9377-bdace3662225 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:38.898427 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 588/1158] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:51:38 2022] GET /identity/v3/users/20d4920e70d74c6ca0fa3e1181ed0edf => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:38.904373 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fabfa75-3b2f-4f25-8a00-402347f71b44 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:38.939585 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:39.013073 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fabfa75-3b2f-4f25-8a00-402347f71b44 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:39.058539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e81187-5707-4c6d-acd4-4349450cb8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:39.059144 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e81187-5707-4c6d-acd4-4349450cb8de None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:39.059422 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e81187-5707-4c6d-acd4-4349450cb8de None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:39.059693 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e81187-5707-4c6d-acd4-4349450cb8de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:39.125002 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e81187-5707-4c6d-acd4-4349450cb8de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:39.126431 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e81187-5707-4c6d-acd4-4349450cb8de None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:39.150663 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:39.225572 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 571/1159] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:51:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:51:39.231821 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb10152-2d0c-40b3-9377-bdace3662225 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:39.331911 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb10152-2d0c-40b3-9377-bdace3662225 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:39.359742 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:39.364985 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ddb31d-12e5-427f-b3e5-b2f9b0db3e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:39.365603 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddb31d-12e5-427f-b3e5-b2f9b0db3e33 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:39.366494 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddb31d-12e5-427f-b3e5-b2f9b0db3e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:39.366766 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddb31d-12e5-427f-b3e5-b2f9b0db3e33 None admin] PATH_INFO: `/v3/projects/7266218fba7042fca5543bd3c34f3420` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:39.367957 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ddb31d-12e5-427f-b3e5-b2f9b0db3e33 None admin] RBAC: Authorizing `identity:get_project(project_id=7266218fba7042fca5543bd3c34f3420)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:39.369406 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ddb31d-12e5-427f-b3e5-b2f9b0db3e33 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:39.371075 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 589/1160] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:51:39 2022] GET /identity/v3/projects/7266218fba7042fca5543bd3c34f3420 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:39.377491 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e81187-5707-4c6d-acd4-4349450cb8de None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:39.474133 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e81187-5707-4c6d-acd4-4349450cb8de None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:39.497708 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:39.510186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9276cc-fcc9-499f-8f0d-ff22a218dbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:39.510789 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9276cc-fcc9-499f-8f0d-ff22a218dbaa None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:39.511070 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9276cc-fcc9-499f-8f0d-ff22a218dbaa None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:39.511347 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9276cc-fcc9-499f-8f0d-ff22a218dbaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:39.512128 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9276cc-fcc9-499f-8f0d-ff22a218dbaa None admin] RBAC: Adding query filter params (role.id=89919f784ad144529b07c8ab92ee7d48, scope.project.id=7266218fba7042fca5543bd3c34f3420, user.id=20d4920e70d74c6ca0fa3e1181ed0edf) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:39.512439 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9276cc-fcc9-499f-8f0d-ff22a218dbaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:39.513886 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9276cc-fcc9-499f-8f0d-ff22a218dbaa None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:39.519613 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 572/1161] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:51:39 2022] GET /identity/v3/role_assignments?user.id=20d4920e70d74c6ca0fa3e1181ed0edf&scope.project.id=7266218fba7042fca5543bd3c34f3420&role.id=89919f784ad144529b07c8ab92ee7d48 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:41.580966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544033a-89cf-461d-8372-88da696c84ee None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:41.581492 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544033a-89cf-461d-8372-88da696c84ee None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:41.581492 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544033a-89cf-461d-8372-88da696c84ee None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:41.582785 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 590/1162] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:41.588520 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f545c-77cf-4e7c-974b-68ef413375dd None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:41.588945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f545c-77cf-4e7c-974b-68ef413375dd None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:41.589099 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f545c-77cf-4e7c-974b-68ef413375dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:41.606691 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-569f545c-77cf-4e7c-974b-68ef413375dd None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:41.607849 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-569f545c-77cf-4e7c-974b-68ef413375dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:41.674115 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 573/1163] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:51:41.697812 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544033a-89cf-461d-8372-88da696c84ee None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:41.800532 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544033a-89cf-461d-8372-88da696c84ee None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:41.840120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9120d3ef-10c1-4e7c-809b-29d60178aef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:41.840688 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120d3ef-10c1-4e7c-809b-29d60178aef9 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:41.841011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120d3ef-10c1-4e7c-809b-29d60178aef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:41.841286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120d3ef-10c1-4e7c-809b-29d60178aef9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:41.906081 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:41.908881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9120d3ef-10c1-4e7c-809b-29d60178aef9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:41.910480 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9120d3ef-10c1-4e7c-809b-29d60178aef9 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:42.008278 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 591/1164] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:51:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:51:42.014263 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f545c-77cf-4e7c-974b-68ef413375dd None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:42.113087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f545c-77cf-4e7c-974b-68ef413375dd None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:42.130940 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:42.156608 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d81a5-b82a-452f-921c-724014357644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:42.157390 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d81a5-b82a-452f-921c-724014357644 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:42.157653 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d81a5-b82a-452f-921c-724014357644 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:42.157923 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d81a5-b82a-452f-921c-724014357644 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:42.165154 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d81a5-b82a-452f-921c-724014357644 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:42.166572 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d81a5-b82a-452f-921c-724014357644 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:42.177823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-e85d81a5-b82a-452f-921c-724014357644 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 05 19:51:42.177823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:42.177823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 05 19:51:42.177823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 05 19:51:42.177823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:51:42.177823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:51:42.177823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:51:42.177823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:51:42.177823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 05 19:51:42.177823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:51:42.177823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:51:42.177823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:51:42.177823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:42.177823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:42.177823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:42.177823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:42.177823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:42.179038 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:42.179038 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:42.179038 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:42.179038 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:42.179038 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:42.179038 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:42.179038 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:51:42.179038 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:51:42.179038 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:51:42.179038 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:51:42.179038 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:42.179038 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:42.179038 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:42.179038 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:42.179038 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:42.179038 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:42.180134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:42.180134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:42.180134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:42.180134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:42.180134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:42.180134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:42.180134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:42.180134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:42.180134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:42.180134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:42.180134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 05 19:51:42.180134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:51:42.180134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 05 19:51:42.180134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:51:42.180134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 574/1165] 10.222.0.39 () {56 vars in 1117 bytes} [Fri Aug 5 19:51:42 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:51:42.185402 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9120d3ef-10c1-4e7c-809b-29d60178aef9 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:42.281013 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:42.284152 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9120d3ef-10c1-4e7c-809b-29d60178aef9 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:42.317742 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640fb63a-6d80-491d-8b49-0d495f2f8f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:42.318348 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640fb63a-6d80-491d-8b49-0d495f2f8f5f None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:42.319155 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640fb63a-6d80-491d-8b49-0d495f2f8f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:42.319742 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640fb63a-6d80-491d-8b49-0d495f2f8f5f None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:42.320126 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640fb63a-6d80-491d-8b49-0d495f2f8f5f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:42.320426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640fb63a-6d80-491d-8b49-0d495f2f8f5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:42.321815 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640fb63a-6d80-491d-8b49-0d495f2f8f5f None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:42.328847 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 592/1166] 10.222.0.39 () {56 vars in 1115 bytes} [Fri Aug 5 19:51:42 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:44.358900 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03d4cb-a55a-4a6c-967b-e016e0a8bbc8 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:44.359451 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03d4cb-a55a-4a6c-967b-e016e0a8bbc8 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:44.359451 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03d4cb-a55a-4a6c-967b-e016e0a8bbc8 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:44.360823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 575/1167] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:44.366658 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209eb091-0016-4670-a19f-1c4ed6a1418d None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:44.367106 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209eb091-0016-4670-a19f-1c4ed6a1418d None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:44.367197 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209eb091-0016-4670-a19f-1c4ed6a1418d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:44.386137 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-209eb091-0016-4670-a19f-1c4ed6a1418d None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:44.387353 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-209eb091-0016-4670-a19f-1c4ed6a1418d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:44.465907 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 593/1168] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:51:44.488338 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b03d4cb-a55a-4a6c-967b-e016e0a8bbc8 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:44.592356 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b03d4cb-a55a-4a6c-967b-e016e0a8bbc8 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:44.626990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b3f56-c682-46e0-8b95-205f8436d1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:44.627602 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b3f56-c682-46e0-8b95-205f8436d1ad None admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:44.627878 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b3f56-c682-46e0-8b95-205f8436d1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:44.628146 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b3f56-c682-46e0-8b95-205f8436d1ad None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:44.628874 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b3f56-c682-46e0-8b95-205f8436d1ad None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:44.630234 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b3f56-c682-46e0-8b95-205f8436d1ad None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:44.646836 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 576/1169] 10.222.0.39 () {60 vars in 1138 bytes} [Fri Aug 5 19:51:44 2022] POST /identity/v3/domains => generated 267 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:51:46.772649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372d064-243d-46ab-8144-f1a928bff121 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:46.773217 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372d064-243d-46ab-8144-f1a928bff121 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:46.773217 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372d064-243d-46ab-8144-f1a928bff121 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:46.774550 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 594/1170] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:51:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:46.780365 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86abd3a7-ffd5-4c4f-abcc-1f23fbe1b1ab None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:46.780805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86abd3a7-ffd5-4c4f-abcc-1f23fbe1b1ab None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:46.780953 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86abd3a7-ffd5-4c4f-abcc-1f23fbe1b1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:46.799405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-86abd3a7-ffd5-4c4f-abcc-1f23fbe1b1ab None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:46.800618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-86abd3a7-ffd5-4c4f-abcc-1f23fbe1b1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:46.831936 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:46.871899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 577/1171] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:51:46.895333 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3372d064-243d-46ab-8144-f1a928bff121 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:46.992598 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:47.000527 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3372d064-243d-46ab-8144-f1a928bff121 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:47.035634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468128c-cfbd-44b1-94bb-5bdb89d3d958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:47.036172 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468128c-cfbd-44b1-94bb-5bdb89d3d958 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:47.036450 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468128c-cfbd-44b1-94bb-5bdb89d3d958 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:47.036751 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468128c-cfbd-44b1-94bb-5bdb89d3d958 None admin] PATH_INFO: `/v3/domains/5628048c73d2412f8c9c0e5b9a18cbfb` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:47.047159 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e468128c-cfbd-44b1-94bb-5bdb89d3d958 None admin] RBAC: Authorizing `identity:get_domain(domain_id=5628048c73d2412f8c9c0e5b9a18cbfb)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:47.048607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e468128c-cfbd-44b1-94bb-5bdb89d3d958 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:47.050217 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 595/1172] 10.222.0.39 () {56 vars in 1183 bytes} [Fri Aug 5 19:51:46 2022] GET /identity/v3/domains/5628048c73d2412f8c9c0e5b9a18cbfb => generated 267 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:47.056669 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86abd3a7-ffd5-4c4f-abcc-1f23fbe1b1ab None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:47.156208 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86abd3a7-ffd5-4c4f-abcc-1f23fbe1b1ab None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:47.190960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de9a84-b8fb-4bae-9f4f-b8993e2b5e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:47.191576 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de9a84-b8fb-4bae-9f4f-b8993e2b5e67 None admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:47.191861 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de9a84-b8fb-4bae-9f4f-b8993e2b5e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:47.192138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de9a84-b8fb-4bae-9f4f-b8993e2b5e67 None admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:47.192850 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de9a84-b8fb-4bae-9f4f-b8993e2b5e67 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:47.194965 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de9a84-b8fb-4bae-9f4f-b8993e2b5e67 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:47.226890 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 578/1173] 10.222.0.39 () {60 vars in 1141 bytes} [Fri Aug 5 19:51:47 2022] POST /identity/v3/projects => generated 361 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:51:49.317761 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17167f3b-add0-4df0-87a7-984615dac0c1 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:49.318482 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17167f3b-add0-4df0-87a7-984615dac0c1 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:49.318482 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17167f3b-add0-4df0-87a7-984615dac0c1 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:49.319844 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 596/1174] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:49.325327 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c8539-2152-40ef-ac0f-451923a6b249 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:49.325824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c8539-2152-40ef-ac0f-451923a6b249 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:49.325928 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c8539-2152-40ef-ac0f-451923a6b249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:49.345498 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-dd3c8539-2152-40ef-ac0f-451923a6b249 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:49.346694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-dd3c8539-2152-40ef-ac0f-451923a6b249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:49.361166 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:49.417774 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 579/1175] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:51:49.441616 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17167f3b-add0-4df0-87a7-984615dac0c1 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:49.548896 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17167f3b-add0-4df0-87a7-984615dac0c1 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:49.585599 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3fd73a-e3ff-40bf-afff-1a0ec1b3b4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:49.586224 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3fd73a-e3ff-40bf-afff-1a0ec1b3b4c2 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:49.586500 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3fd73a-e3ff-40bf-afff-1a0ec1b3b4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:49.586770 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3fd73a-e3ff-40bf-afff-1a0ec1b3b4c2 None admin] PATH_INFO: `/v3/domains/5628048c73d2412f8c9c0e5b9a18cbfb` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:49.587922 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3fd73a-e3ff-40bf-afff-1a0ec1b3b4c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=5628048c73d2412f8c9c0e5b9a18cbfb)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:49.589359 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3fd73a-e3ff-40bf-afff-1a0ec1b3b4c2 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:49.590943 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 597/1176] 10.222.0.39 () {56 vars in 1183 bytes} [Fri Aug 5 19:51:49 2022] GET /identity/v3/domains/5628048c73d2412f8c9c0e5b9a18cbfb => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:49.596829 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c8539-2152-40ef-ac0f-451923a6b249 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:49.698084 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c8539-2152-40ef-ac0f-451923a6b249 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:49.733384 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462dd93c-4c01-465b-b44b-27acc55d2abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:49.734029 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462dd93c-4c01-465b-b44b-27acc55d2abb None admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:49.734312 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462dd93c-4c01-465b-b44b-27acc55d2abb None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:49.734580 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462dd93c-4c01-465b-b44b-27acc55d2abb None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:49.735262 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462dd93c-4c01-465b-b44b-27acc55d2abb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:49.736830 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462dd93c-4c01-465b-b44b-27acc55d2abb None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:49.761618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 580/1177] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Aug 5 19:51:49 2022] POST /identity/v3/users => generated 311 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:51:51.873927 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcfd9c-aee8-4c47-a515-c53499f96b4c None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:51.874457 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcfd9c-aee8-4c47-a515-c53499f96b4c None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:51.874457 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcfd9c-aee8-4c47-a515-c53499f96b4c None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:51.875831 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 598/1178] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:51.881721 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9df926-025c-44cd-b907-35a106443cec None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:51.882180 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9df926-025c-44cd-b907-35a106443cec None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:51.882275 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9df926-025c-44cd-b907-35a106443cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:51.889398 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:51.902885 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-9f9df926-025c-44cd-b907-35a106443cec None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:51.904066 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-9f9df926-025c-44cd-b907-35a106443cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:51.975359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 581/1179] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:51:52.003013 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbcfd9c-aee8-4c47-a515-c53499f96b4c None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:52.028965 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:52.105434 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbcfd9c-aee8-4c47-a515-c53499f96b4c None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:52.139124 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a385c7e-b030-484f-b179-aec5e186c4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:52.139768 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a385c7e-b030-484f-b179-aec5e186c4cc None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:52.140052 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a385c7e-b030-484f-b179-aec5e186c4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:52.140324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a385c7e-b030-484f-b179-aec5e186c4cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:52.152988 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a385c7e-b030-484f-b179-aec5e186c4cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:51:52.157867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-6a385c7e-b030-484f-b179-aec5e186c4cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:51:52.157867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:52.157867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:52.157867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:52.157867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:52.157867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:52.157867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:52.157867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:52.157867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:52.157867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:52.157867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:52.157867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:52.157867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:51:52.157867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:51:52.157867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:51:52.157867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:51:52.158966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:51:52.158966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:51:52.158966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:51:52.158966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:51:52.158966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:52.158966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:52.158966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:52.158966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:52.158966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:52.158966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:52.158966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:52.158966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:52.158966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:52.158966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:52.158966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:52.158966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:52.160008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:52.160008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:52.160008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:52.160008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:52.160008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:51:52.160008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:51:52.160008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:51:52.160008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:51:52.160008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:51:52.160008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:51:52.160008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:51:52.160008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:51:52.160008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 599/1180] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:51:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:51:52.165413 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9df926-025c-44cd-b907-35a106443cec None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:52.267831 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9df926-025c-44cd-b907-35a106443cec None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:52.302540 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a4c33-0c0a-4224-b1d7-cf1a0246a308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:52.303162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a4c33-0c0a-4224-b1d7-cf1a0246a308 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:52.303480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a4c33-0c0a-4224-b1d7-cf1a0246a308 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:52.303753 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a4c33-0c0a-4224-b1d7-cf1a0246a308 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:52.304473 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a4c33-0c0a-4224-b1d7-cf1a0246a308 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:52.304901 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a4c33-0c0a-4224-b1d7-cf1a0246a308 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:52.306168 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a4c33-0c0a-4224-b1d7-cf1a0246a308 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:52.307602 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:52.314924 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 582/1181] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:51:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:52.321498 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a385c7e-b030-484f-b179-aec5e186c4cc None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:52.411318 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:52.430420 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a385c7e-b030-484f-b179-aec5e186c4cc None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:52.464305 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c54b5-daf1-4dae-ac2a-ad2f11ff7f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:52.464968 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c54b5-daf1-4dae-ac2a-ad2f11ff7f5b None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:52.465252 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c54b5-daf1-4dae-ac2a-ad2f11ff7f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:52.465529 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c54b5-daf1-4dae-ac2a-ad2f11ff7f5b None admin] PATH_INFO: `/v3/users/8dbe31301f7940d4b0e3b2e75742927a` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:52.478416 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705c54b5-daf1-4dae-ac2a-ad2f11ff7f5b None admin] RBAC: Authorizing `identity:get_user(user_id=8dbe31301f7940d4b0e3b2e75742927a)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:52.479853 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705c54b5-daf1-4dae-ac2a-ad2f11ff7f5b None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:52.481526 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 600/1182] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:51:52 2022] GET /identity/v3/users/8dbe31301f7940d4b0e3b2e75742927a => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:52.488419 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a4c33-0c0a-4224-b1d7-cf1a0246a308 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:52.588383 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a4c33-0c0a-4224-b1d7-cf1a0246a308 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:52.624075 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deafaa5-c463-4928-9d82-a3fdc1c85926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:52.624726 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deafaa5-c463-4928-9d82-a3fdc1c85926 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:52.625008 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deafaa5-c463-4928-9d82-a3fdc1c85926 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:52.625284 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deafaa5-c463-4928-9d82-a3fdc1c85926 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:52.687815 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:52.695461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deafaa5-c463-4928-9d82-a3fdc1c85926 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:52.696953 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deafaa5-c463-4928-9d82-a3fdc1c85926 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:52.800222 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 583/1183] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:51:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:51:52.807109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c54b5-daf1-4dae-ac2a-ad2f11ff7f5b None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:52.911446 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c54b5-daf1-4dae-ac2a-ad2f11ff7f5b None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:52.946793 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96e22a3-9ac0-4180-b2ac-623e54b04d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:52.947347 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e22a3-9ac0-4180-b2ac-623e54b04d5b None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:52.947629 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e22a3-9ac0-4180-b2ac-623e54b04d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:52.947904 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e22a3-9ac0-4180-b2ac-623e54b04d5b None admin] PATH_INFO: `/v3/projects/50e957a3b034448aace6a8bbcac0c1f5` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:52.949255 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96e22a3-9ac0-4180-b2ac-623e54b04d5b None admin] RBAC: Authorizing `identity:get_project(project_id=50e957a3b034448aace6a8bbcac0c1f5)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:52.950768 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96e22a3-9ac0-4180-b2ac-623e54b04d5b None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:52.952476 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 601/1184] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:51:52 2022] GET /identity/v3/projects/50e957a3b034448aace6a8bbcac0c1f5 => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:52.959244 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deafaa5-c463-4928-9d82-a3fdc1c85926 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:53.056293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:53.062592 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deafaa5-c463-4928-9d82-a3fdc1c85926 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:53.097381 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2895aa1e-3c5e-433f-9124-c221eb4c4cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:53.097997 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895aa1e-3c5e-433f-9124-c221eb4c4cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:53.098281 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895aa1e-3c5e-433f-9124-c221eb4c4cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:53.098560 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895aa1e-3c5e-433f-9124-c221eb4c4cf6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:53.100113 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2895aa1e-3c5e-433f-9124-c221eb4c4cf6 None admin] RBAC: Adding query filter params (role.id=89919f784ad144529b07c8ab92ee7d48, scope.project.id=50e957a3b034448aace6a8bbcac0c1f5, user.id=8dbe31301f7940d4b0e3b2e75742927a) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:53.100430 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2895aa1e-3c5e-433f-9124-c221eb4c4cf6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:53.101896 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2895aa1e-3c5e-433f-9124-c221eb4c4cf6 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:53.107560 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 584/1185] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:51:52 2022] GET /identity/v3/role_assignments?user.id=8dbe31301f7940d4b0e3b2e75742927a&scope.project.id=50e957a3b034448aace6a8bbcac0c1f5&role.id=89919f784ad144529b07c8ab92ee7d48 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:55.101117 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59b8e4-e9e0-4ec8-be24-dc52f17f9b23 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:55.101684 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59b8e4-e9e0-4ec8-be24-dc52f17f9b23 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:55.101684 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59b8e4-e9e0-4ec8-be24-dc52f17f9b23 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:55.103002 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 602/1186] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:51:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:55.108144 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb3cba-379c-4458-9ae4-352e76b61760 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:55.108563 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb3cba-379c-4458-9ae4-352e76b61760 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:55.108723 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb3cba-379c-4458-9ae4-352e76b61760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:55.127872 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-0bfb3cba-379c-4458-9ae4-352e76b61760 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:55.129064 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-0bfb3cba-379c-4458-9ae4-352e76b61760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:55.198003 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 585/1187] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:51:55.220222 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59b8e4-e9e0-4ec8-be24-dc52f17f9b23 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:55.231745 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:55.322894 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59b8e4-e9e0-4ec8-be24-dc52f17f9b23 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:55.356012 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264988b0-03bd-4628-9324-3ad4fb43d836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:55.356562 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264988b0-03bd-4628-9324-3ad4fb43d836 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:55.356926 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264988b0-03bd-4628-9324-3ad4fb43d836 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:55.357222 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264988b0-03bd-4628-9324-3ad4fb43d836 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:55.367208 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264988b0-03bd-4628-9324-3ad4fb43d836 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:51:55.372110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-264988b0-03bd-4628-9324-3ad4fb43d836 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:51:55.372110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:55.372110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:55.372110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:55.372110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:55.372110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:55.372110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:55.372110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:55.372110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:55.372110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:55.372110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:55.372110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:55.372110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:51:55.372110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:51:55.372110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:51:55.372110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:51:55.373286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:51:55.373286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:51:55.373286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:51:55.373286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:51:55.373286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:55.373286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:55.373286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:55.373286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:55.373286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:55.373286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:55.373286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:55.373286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:55.373286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:55.373286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:55.373286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:55.373286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:55.374525 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:55.374525 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:55.374525 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:55.374525 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:55.374525 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:51:55.374525 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:51:55.374525 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:51:55.374525 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:51:55.374525 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:51:55.374525 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:51:55.374525 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:51:55.374525 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:51:55.374525 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 603/1188] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:51:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:51:55.379366 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb3cba-379c-4458-9ae4-352e76b61760 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:55.467661 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:55.477939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb3cba-379c-4458-9ae4-352e76b61760 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:55.511620 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdce0a8-f709-4aff-893f-ce24176622ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:55.512226 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdce0a8-f709-4aff-893f-ce24176622ba None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:55.512504 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdce0a8-f709-4aff-893f-ce24176622ba None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:55.512802 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdce0a8-f709-4aff-893f-ce24176622ba None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:55.513504 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdce0a8-f709-4aff-893f-ce24176622ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:55.513805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdce0a8-f709-4aff-893f-ce24176622ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:55.515146 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdce0a8-f709-4aff-893f-ce24176622ba None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:55.523449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 586/1189] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:51:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:55.529515 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264988b0-03bd-4628-9324-3ad4fb43d836 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:55.596919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:55.632517 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264988b0-03bd-4628-9324-3ad4fb43d836 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:55.666021 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8323e-17e6-4c01-b2ef-24c4674ecfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:55.666740 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8323e-17e6-4c01-b2ef-24c4674ecfca None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:55.667058 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8323e-17e6-4c01-b2ef-24c4674ecfca None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:55.667379 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8323e-17e6-4c01-b2ef-24c4674ecfca None admin] PATH_INFO: `/v3/users/8dbe31301f7940d4b0e3b2e75742927a` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:55.668546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c8323e-17e6-4c01-b2ef-24c4674ecfca None admin] RBAC: Authorizing `identity:get_user(user_id=8dbe31301f7940d4b0e3b2e75742927a)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:55.670944 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c8323e-17e6-4c01-b2ef-24c4674ecfca None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:55.673556 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 604/1190] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:51:55 2022] GET /identity/v3/users/8dbe31301f7940d4b0e3b2e75742927a => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:55.680159 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdce0a8-f709-4aff-893f-ce24176622ba None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:55.780147 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdce0a8-f709-4aff-893f-ce24176622ba None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:55.814413 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d415c6-23a4-40eb-b8a2-b7dabf1c1ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:55.815066 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d415c6-23a4-40eb-b8a2-b7dabf1c1ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:55.815345 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d415c6-23a4-40eb-b8a2-b7dabf1c1ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:55.815619 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d415c6-23a4-40eb-b8a2-b7dabf1c1ea7 None admin] PATH_INFO: `/v3/projects/50e957a3b034448aace6a8bbcac0c1f5` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:55.816782 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d415c6-23a4-40eb-b8a2-b7dabf1c1ea7 None admin] RBAC: Authorizing `identity:get_project(project_id=50e957a3b034448aace6a8bbcac0c1f5)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:55.819082 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d415c6-23a4-40eb-b8a2-b7dabf1c1ea7 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:55.820887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 587/1191] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:51:55 2022] GET /identity/v3/projects/50e957a3b034448aace6a8bbcac0c1f5 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:55.826936 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8323e-17e6-4c01-b2ef-24c4674ecfca None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:55.926276 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8323e-17e6-4c01-b2ef-24c4674ecfca None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:55.960875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b016c0-ae64-4c58-90e0-3f3209c257d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:55.961530 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b016c0-ae64-4c58-90e0-3f3209c257d3 None admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:55.961807 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b016c0-ae64-4c58-90e0-3f3209c257d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:55.962083 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b016c0-ae64-4c58-90e0-3f3209c257d3 None admin] PATH_INFO: `/v3/projects/50e957a3b034448aace6a8bbcac0c1f5/users/8dbe31301f7940d4b0e3b2e75742927a/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:55.963966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b016c0-ae64-4c58-90e0-3f3209c257d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=50e957a3b034448aace6a8bbcac0c1f5, user_id=8dbe31301f7940d4b0e3b2e75742927a, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:55.965408 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b016c0-ae64-4c58-90e0-3f3209c257d3 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:55.968405 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:55.979547 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 605/1192] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Aug 5 19:51:55 2022] PUT /identity/v3/projects/50e957a3b034448aace6a8bbcac0c1f5/users/8dbe31301f7940d4b0e3b2e75742927a/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:58.069198 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a44335-f269-48b6-a9ca-3a77747fa590 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:58.069780 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a44335-f269-48b6-a9ca-3a77747fa590 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:58.069961 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a44335-f269-48b6-a9ca-3a77747fa590 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:58.071882 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 588/1193] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:51:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:58.077247 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99503f3d-ed37-49e7-b413-09ebd8019957 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:58.077707 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99503f3d-ed37-49e7-b413-09ebd8019957 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:58.077810 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99503f3d-ed37-49e7-b413-09ebd8019957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:58.097907 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-99503f3d-ed37-49e7-b413-09ebd8019957 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:58.099616 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-99503f3d-ed37-49e7-b413-09ebd8019957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:58.178804 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 606/1194] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:51:58.203190 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a44335-f269-48b6-a9ca-3a77747fa590 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:58.226212 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:58.328921 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a44335-f269-48b6-a9ca-3a77747fa590 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:58.362080 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43221e-cd69-44b8-8d8a-639df41d969a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:58.362715 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43221e-cd69-44b8-8d8a-639df41d969a None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:58.362994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43221e-cd69-44b8-8d8a-639df41d969a None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:58.363265 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43221e-cd69-44b8-8d8a-639df41d969a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:58.374302 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a43221e-cd69-44b8-8d8a-639df41d969a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:51:58.379130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-9a43221e-cd69-44b8-8d8a-639df41d969a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:51:58.379130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:58.379130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:58.379130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:58.379130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:58.379130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:58.379130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:58.379130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:58.379130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:58.379130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:58.379130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:58.379130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:58.379130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:51:58.379130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:51:58.379130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:51:58.379130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:51:58.380350 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:51:58.380350 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:51:58.380350 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:51:58.380350 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:51:58.380350 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:58.380350 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:58.380350 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:58.380350 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:58.380350 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:58.380350 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:58.380350 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:58.380350 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:58.380350 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:58.380350 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:58.380350 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:58.380350 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:58.381468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:58.381468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:58.381468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:58.381468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:58.381468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:51:58.381468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:51:58.381468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:51:58.381468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:51:58.381468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:51:58.381468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:51:58.381468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:51:58.381468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:51:58.381468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 589/1195] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:51:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:51:58.388585 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99503f3d-ed37-49e7-b413-09ebd8019957 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:58.486866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99503f3d-ed37-49e7-b413-09ebd8019957 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:58.511260 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:58.523272 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc5d18-8582-4d44-b222-78c1ce96d8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:58.523879 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc5d18-8582-4d44-b222-78c1ce96d8b1 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:58.524153 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc5d18-8582-4d44-b222-78c1ce96d8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:58.524419 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc5d18-8582-4d44-b222-78c1ce96d8b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:58.525157 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc5d18-8582-4d44-b222-78c1ce96d8b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:58.525455 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc5d18-8582-4d44-b222-78c1ce96d8b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:58.526815 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc5d18-8582-4d44-b222-78c1ce96d8b1 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:58.535004 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 607/1196] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:51:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:58.540956 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43221e-cd69-44b8-8d8a-639df41d969a None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:58.622216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:58.641886 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43221e-cd69-44b8-8d8a-639df41d969a None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:58.674893 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf2ec6-0706-4e74-a043-50c4158b93a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:58.675514 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf2ec6-0706-4e74-a043-50c4158b93a4 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:58.675788 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf2ec6-0706-4e74-a043-50c4158b93a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:58.676057 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf2ec6-0706-4e74-a043-50c4158b93a4 None admin] PATH_INFO: `/v3/users/8dbe31301f7940d4b0e3b2e75742927a` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:58.677734 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf2ec6-0706-4e74-a043-50c4158b93a4 None admin] RBAC: Authorizing `identity:get_user(user_id=8dbe31301f7940d4b0e3b2e75742927a)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:58.679305 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf2ec6-0706-4e74-a043-50c4158b93a4 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:58.680929 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 590/1197] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Aug 5 19:51:58 2022] GET /identity/v3/users/8dbe31301f7940d4b0e3b2e75742927a => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:51:58.695725 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc5d18-8582-4d44-b222-78c1ce96d8b1 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:58.795639 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc5d18-8582-4d44-b222-78c1ce96d8b1 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:58.828927 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a22fa6-3847-45b1-a7af-36a342e8bee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:58.829540 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a22fa6-3847-45b1-a7af-36a342e8bee9 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:58.829822 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a22fa6-3847-45b1-a7af-36a342e8bee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:58.830098 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a22fa6-3847-45b1-a7af-36a342e8bee9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:58.885943 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:58.898814 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a22fa6-3847-45b1-a7af-36a342e8bee9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:58.900250 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a22fa6-3847-45b1-a7af-36a342e8bee9 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:59.002254 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 608/1198] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:51:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:51:59.009014 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf2ec6-0706-4e74-a043-50c4158b93a4 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:59.110900 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf2ec6-0706-4e74-a043-50c4158b93a4 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:59.148219 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a188cc0-b0f7-42f7-bb73-41ddbafcbfc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:59.148883 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a188cc0-b0f7-42f7-bb73-41ddbafcbfc0 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:59.149162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a188cc0-b0f7-42f7-bb73-41ddbafcbfc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:59.149434 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a188cc0-b0f7-42f7-bb73-41ddbafcbfc0 None admin] PATH_INFO: `/v3/projects/50e957a3b034448aace6a8bbcac0c1f5` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:59.150561 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a188cc0-b0f7-42f7-bb73-41ddbafcbfc0 None admin] RBAC: Authorizing `identity:get_project(project_id=50e957a3b034448aace6a8bbcac0c1f5)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:59.151976 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a188cc0-b0f7-42f7-bb73-41ddbafcbfc0 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:59.153647 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 591/1199] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Aug 5 19:51:59 2022] GET /identity/v3/projects/50e957a3b034448aace6a8bbcac0c1f5 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:59.159664 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a22fa6-3847-45b1-a7af-36a342e8bee9 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:59.244346 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:59.257386 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a22fa6-3847-45b1-a7af-36a342e8bee9 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:59.291152 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889ec3ea-28d5-49f4-b7db-cc34ef28fc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:59.291755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ec3ea-28d5-49f4-b7db-cc34ef28fc79 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:59.292035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ec3ea-28d5-49f4-b7db-cc34ef28fc79 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:59.292307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ec3ea-28d5-49f4-b7db-cc34ef28fc79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:59.293118 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889ec3ea-28d5-49f4-b7db-cc34ef28fc79 None admin] RBAC: Adding query filter params (role.id=89919f784ad144529b07c8ab92ee7d48, scope.project.id=50e957a3b034448aace6a8bbcac0c1f5, user.id=8dbe31301f7940d4b0e3b2e75742927a) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:59.293428 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889ec3ea-28d5-49f4-b7db-cc34ef28fc79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:59.294837 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889ec3ea-28d5-49f4-b7db-cc34ef28fc79 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:59.300468 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 609/1200] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:51:59 2022] GET /identity/v3/role_assignments?user.id=8dbe31301f7940d4b0e3b2e75742927a&scope.project.id=50e957a3b034448aace6a8bbcac0c1f5&role.id=89919f784ad144529b07c8ab92ee7d48 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:52:09.143089 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=92 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:09.297915 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=24 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:55.794639 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407244b1-45e5-41a5-9569-c20ec3be1dc8 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:55.795267 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407244b1-45e5-41a5-9569-c20ec3be1dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:55.795267 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407244b1-45e5-41a5-9569-c20ec3be1dc8 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:55.796880 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 592/1201] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:52:55.803711 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2609606-b416-4ca6-84a8-35c3c869f19a None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:55.804406 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2609606-b416-4ca6-84a8-35c3c869f19a None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:55.804406 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2609606-b416-4ca6-84a8-35c3c869f19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:55.827441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-d2609606-b416-4ca6-84a8-35c3c869f19a None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:55.829635 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-d2609606-b416-4ca6-84a8-35c3c869f19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:55.928761 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 610/1202] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:52:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:52:55.947260 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407244b1-45e5-41a5-9569-c20ec3be1dc8 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:56.095654 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407244b1-45e5-41a5-9569-c20ec3be1dc8 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:56.146005 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce84aa-7076-4d16-898b-4ba16872686c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:56.146659 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce84aa-7076-4d16-898b-4ba16872686c None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:56.146942 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce84aa-7076-4d16-898b-4ba16872686c None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:56.147212 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce84aa-7076-4d16-898b-4ba16872686c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:56.162974 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce84aa-7076-4d16-898b-4ba16872686c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:56.164567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce84aa-7076-4d16-898b-4ba16872686c None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:56.182240 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-90ce84aa-7076-4d16-898b-4ba16872686c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:52:56.182240 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:52:56.182240 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:52:56.182240 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:52:56.182240 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:52:56.182240 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:52:56.182240 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:52:56.182240 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:52:56.182240 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:52:56.182240 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:52:56.182240 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:52:56.182240 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:52:56.182240 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:52:56.182240 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:52:56.182240 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:52:56.183372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:52:56.183372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:52:56.183372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:52:56.183372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:52:56.183372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:52:56.183372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:52:56.183372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:52:56.183372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:52:56.183372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:52:56.183372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:52:56.183372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:52:56.183372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:52:56.183372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:52:56.183372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:52:56.183372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:52:56.183372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:52:56.183372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:52:56.184599 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:52:56.184599 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:52:56.184599 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:52:56.184599 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:52:56.184599 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 593/1203] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:52:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:52:56.190346 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2609606-b416-4ca6-84a8-35c3c869f19a None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:56.326153 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2609606-b416-4ca6-84a8-35c3c869f19a None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:56.362336 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:56.370369 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b5a3c-8b83-4e2c-af1f-9e909f191da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:56.371450 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b5a3c-8b83-4e2c-af1f-9e909f191da5 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:56.371929 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b5a3c-8b83-4e2c-af1f-9e909f191da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:56.372401 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b5a3c-8b83-4e2c-af1f-9e909f191da5 None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:56.373385 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b5a3c-8b83-4e2c-af1f-9e909f191da5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:52:56.373719 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b5a3c-8b83-4e2c-af1f-9e909f191da5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:56.376469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b5a3c-8b83-4e2c-af1f-9e909f191da5 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:56.396530 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 611/1204] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:52:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:52:56.405587 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce84aa-7076-4d16-898b-4ba16872686c None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:56.472510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:56.540130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce84aa-7076-4d16-898b-4ba16872686c None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:56.586998 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a458b8-e97f-4fc9-ad13-5ecb533f4f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:56.588011 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a458b8-e97f-4fc9-ad13-5ecb533f4f78 None admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:56.588286 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a458b8-e97f-4fc9-ad13-5ecb533f4f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:56.588606 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a458b8-e97f-4fc9-ad13-5ecb533f4f78 None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:56.589356 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a458b8-e97f-4fc9-ad13-5ecb533f4f78 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:56.590801 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a458b8-e97f-4fc9-ad13-5ecb533f4f78 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:56.617041 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 594/1205] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Aug 5 19:52:56 2022] POST /identity/v3/users => generated 251 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:52:58.959409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b291c2-4c33-4cf3-b223-2a47406eb05d None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:58.960014 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b291c2-4c33-4cf3-b223-2a47406eb05d None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:58.960014 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b291c2-4c33-4cf3-b223-2a47406eb05d None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:58.961745 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 612/1206] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:52:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:52:58.969962 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365443f-24ad-4b51-8835-848cc8a9fc49 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:58.975655 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365443f-24ad-4b51-8835-848cc8a9fc49 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:58.975655 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365443f-24ad-4b51-8835-848cc8a9fc49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:58.994179 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-7365443f-24ad-4b51-8835-848cc8a9fc49 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:58.995347 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-7365443f-24ad-4b51-8835-848cc8a9fc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:59.082978 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 595/1207] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:52:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:52:59.112352 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b291c2-4c33-4cf3-b223-2a47406eb05d None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:59.257791 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b291c2-4c33-4cf3-b223-2a47406eb05d None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:59.302477 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dcd8ed-a7ba-4997-91f9-3ebf1e6058fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:59.302997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcd8ed-a7ba-4997-91f9-3ebf1e6058fb None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:59.303367 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcd8ed-a7ba-4997-91f9-3ebf1e6058fb None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:59.303681 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcd8ed-a7ba-4997-91f9-3ebf1e6058fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:59.319813 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dcd8ed-a7ba-4997-91f9-3ebf1e6058fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:52:59.327215 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-c2dcd8ed-a7ba-4997-91f9-3ebf1e6058fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:52:59.327215 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:52:59.327215 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:52:59.327215 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:52:59.327215 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:52:59.327215 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:52:59.327215 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:52:59.327215 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:52:59.327215 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:52:59.327215 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:52:59.327215 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:52:59.327215 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:52:59.327215 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:52:59.327215 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:52:59.327215 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:52:59.327215 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:52:59.328276 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:52:59.328276 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:52:59.328276 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:52:59.328276 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:52:59.328276 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:52:59.328276 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:52:59.328276 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:52:59.328276 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:52:59.328276 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:52:59.328276 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:52:59.328276 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:52:59.328276 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:52:59.328276 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:52:59.328276 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:52:59.328276 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:52:59.328276 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:52:59.330994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:52:59.330994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:52:59.330994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:52:59.330994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:52:59.330994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:52:59.330994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:52:59.330994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:52:59.330994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:52:59.330994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:52:59.330994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:52:59.330994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:52:59.330994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:52:59.336444 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 613/1208] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Aug 5 19:52:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:52:59.336638 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365443f-24ad-4b51-8835-848cc8a9fc49 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:59.383668 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:59.478896 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365443f-24ad-4b51-8835-848cc8a9fc49 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:59.538274 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f3f3e-a5b9-47a0-a070-5771a2a90062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:59.539557 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f3f3e-a5b9-47a0-a070-5771a2a90062 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:59.539973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f3f3e-a5b9-47a0-a070-5771a2a90062 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:59.540561 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f3f3e-a5b9-47a0-a070-5771a2a90062 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:59.541576 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421f3f3e-a5b9-47a0-a070-5771a2a90062 None admin] RBAC: Adding query filter params (name=service) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:52:59.542167 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421f3f3e-a5b9-47a0-a070-5771a2a90062 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:59.543856 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421f3f3e-a5b9-47a0-a070-5771a2a90062 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:59.559843 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 596/1209] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Aug 5 19:52:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:52:59.567190 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dcd8ed-a7ba-4997-91f9-3ebf1e6058fb None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:59.650667 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:59.739529 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dcd8ed-a7ba-4997-91f9-3ebf1e6058fb None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:59.789455 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d9c8c5-2833-4d36-be86-9cdd86b89555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:59.790022 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d9c8c5-2833-4d36-be86-9cdd86b89555 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:59.790305 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d9c8c5-2833-4d36-be86-9cdd86b89555 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:59.790602 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d9c8c5-2833-4d36-be86-9cdd86b89555 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:59.803382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d9c8c5-2833-4d36-be86-9cdd86b89555 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:59.805705 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d9c8c5-2833-4d36-be86-9cdd86b89555 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:59.823773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-95d9c8c5-2833-4d36-be86-9cdd86b89555 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:52:59.823773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:52:59.823773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:52:59.823773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:52:59.823773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:52:59.823773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:52:59.823773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:52:59.823773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:52:59.823773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:52:59.823773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:52:59.823773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:52:59.823773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:52:59.823773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:52:59.823773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:52:59.823773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:52:59.825041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:52:59.825041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:52:59.825041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:52:59.825041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:52:59.825041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:52:59.825041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:52:59.825041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:52:59.825041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:52:59.825041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:52:59.825041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:52:59.825041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:52:59.825041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:52:59.825041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:52:59.825041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:52:59.825041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:52:59.825041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:52:59.825041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:52:59.826620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:52:59.826620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:52:59.826620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:52:59.826620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:52:59.826620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 614/1210] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:52:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:52:59.832518 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f3f3e-a5b9-47a0-a070-5771a2a90062 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:59.967352 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:59.971350 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f3f3e-a5b9-47a0-a070-5771a2a90062 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:00.018258 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2388903-0275-4477-858e-c0e702729437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:00.019196 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2388903-0275-4477-858e-c0e702729437 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:00.019529 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2388903-0275-4477-858e-c0e702729437 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:00.019883 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2388903-0275-4477-858e-c0e702729437 None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:00.020827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2388903-0275-4477-858e-c0e702729437 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:00.021195 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2388903-0275-4477-858e-c0e702729437 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:00.022719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2388903-0275-4477-858e-c0e702729437 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:00.036551 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 597/1211] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:52:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:00.042765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d9c8c5-2833-4d36-be86-9cdd86b89555 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:00.180361 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d9c8c5-2833-4d36-be86-9cdd86b89555 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:00.184808 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:00.229884 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e046e1-4fd2-4a99-a4a7-555b6f8ee257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:00.230735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e046e1-4fd2-4a99-a4a7-555b6f8ee257 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:00.231066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e046e1-4fd2-4a99-a4a7-555b6f8ee257 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:00.231394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e046e1-4fd2-4a99-a4a7-555b6f8ee257 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:00.243452 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e046e1-4fd2-4a99-a4a7-555b6f8ee257 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:00.245255 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e046e1-4fd2-4a99-a4a7-555b6f8ee257 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:00.259288 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-d9e046e1-4fd2-4a99-a4a7-555b6f8ee257 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 19:53:00.259288 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:00.259288 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:53:00.259288 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:53:00.259288 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:53:00.259288 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:53:00.259288 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:53:00.259288 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:53:00.259288 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:53:00.259288 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:53:00.259288 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:53:00.259288 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:53:00.259288 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:53:00.259288 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:53:00.259288 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:53:00.259288 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:53:00.260948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:53:00.260948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:53:00.260948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:53:00.260948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:53:00.260948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:53:00.260948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:53:00.260948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:53:00.260948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:53:00.260948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:53:00.260948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:53:00.260948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:53:00.260948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:53:00.260948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:53:00.260948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:53:00.260948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:53:00.260948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:53:00.260948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:53:00.260948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:53:00.262400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:53:00.262400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:53:00.262400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:53:00.262400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:53:00.262400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:53:00.262400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:53:00.262400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:53:00.262400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:53:00.262400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 19:53:00.262400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:53:00.268744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 615/1212] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:53:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:53:00.270909 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2388903-0275-4477-858e-c0e702729437 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:00.409895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2388903-0275-4477-858e-c0e702729437 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:00.460334 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d89a77-f6c4-4c2d-8fe4-2d0ccac165dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:00.460984 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d89a77-f6c4-4c2d-8fe4-2d0ccac165dd None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:00.461265 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d89a77-f6c4-4c2d-8fe4-2d0ccac165dd None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:00.461535 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d89a77-f6c4-4c2d-8fe4-2d0ccac165dd None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:00.462403 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d89a77-f6c4-4c2d-8fe4-2d0ccac165dd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:00.462643 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d89a77-f6c4-4c2d-8fe4-2d0ccac165dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:00.464065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d89a77-f6c4-4c2d-8fe4-2d0ccac165dd None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:00.482945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 598/1213] 10.222.0.39 () {56 vars in 1143 bytes} [Fri Aug 5 19:53:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:00.489723 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e046e1-4fd2-4a99-a4a7-555b6f8ee257 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:00.627524 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e046e1-4fd2-4a99-a4a7-555b6f8ee257 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:00.669300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60632ffe-69bc-4b9c-8e6a-4b1bcd2666ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:00.669877 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60632ffe-69bc-4b9c-8e6a-4b1bcd2666ac None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:00.670126 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60632ffe-69bc-4b9c-8e6a-4b1bcd2666ac None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:00.670445 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60632ffe-69bc-4b9c-8e6a-4b1bcd2666ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:00.707467 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:00.774821 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60632ffe-69bc-4b9c-8e6a-4b1bcd2666ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:00.776301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60632ffe-69bc-4b9c-8e6a-4b1bcd2666ac None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:00.904994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 616/1214] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:53:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 19:53:00.914399 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d89a77-f6c4-4c2d-8fe4-2d0ccac165dd None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:00.936631 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:01.038979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d89a77-f6c4-4c2d-8fe4-2d0ccac165dd None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:01.084344 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b3bba-ba84-4162-a318-00e83d93c532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:01.085016 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b3bba-ba84-4162-a318-00e83d93c532 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:01.085363 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b3bba-ba84-4162-a318-00e83d93c532 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:01.085744 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b3bba-ba84-4162-a318-00e83d93c532 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:01.100330 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b3bba-ba84-4162-a318-00e83d93c532 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:01.103178 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b3bba-ba84-4162-a318-00e83d93c532 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:01.126573 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-5a6b3bba-ba84-4162-a318-00e83d93c532 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:53:01.126573 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:01.126573 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:53:01.126573 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:53:01.126573 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:53:01.126573 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:53:01.126573 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:53:01.126573 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:53:01.126573 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:53:01.126573 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:53:01.126573 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:53:01.126573 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:53:01.126573 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:53:01.126573 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:53:01.126573 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:53:01.127919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:53:01.127919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:53:01.127919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:53:01.127919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:53:01.127919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:53:01.127919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:53:01.127919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:53:01.127919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:53:01.127919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:53:01.127919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:53:01.127919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:53:01.127919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:53:01.127919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:53:01.127919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:53:01.127919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:53:01.127919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:53:01.127919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:53:01.129030 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:53:01.129030 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:53:01.129030 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:53:01.129030 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:53:01.129030 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 599/1215] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:53:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:53:01.134589 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60632ffe-69bc-4b9c-8e6a-4b1bcd2666ac None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:01.212554 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:01.261720 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60632ffe-69bc-4b9c-8e6a-4b1bcd2666ac None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:01.303711 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50f588-53e2-40c8-bd81-0b24f554e119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:01.304333 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50f588-53e2-40c8-bd81-0b24f554e119 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:01.304753 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50f588-53e2-40c8-bd81-0b24f554e119 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:01.305163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50f588-53e2-40c8-bd81-0b24f554e119 None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:01.306197 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50f588-53e2-40c8-bd81-0b24f554e119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:01.306631 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50f588-53e2-40c8-bd81-0b24f554e119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:01.308419 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50f588-53e2-40c8-bd81-0b24f554e119 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:01.321922 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 617/1216] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:53:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:01.329208 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b3bba-ba84-4162-a318-00e83d93c532 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:01.420459 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:01.461248 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b3bba-ba84-4162-a318-00e83d93c532 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:01.503251 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54abf13-8a84-4797-aaea-fd9f00eb39ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:01.504301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54abf13-8a84-4797-aaea-fd9f00eb39ce None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:01.504821 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54abf13-8a84-4797-aaea-fd9f00eb39ce None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:01.505350 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54abf13-8a84-4797-aaea-fd9f00eb39ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:01.513972 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54abf13-8a84-4797-aaea-fd9f00eb39ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:01.515787 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54abf13-8a84-4797-aaea-fd9f00eb39ce None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:01.530567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-f54abf13-8a84-4797-aaea-fd9f00eb39ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:53:01.530567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:01.530567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:53:01.530567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:53:01.530567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:53:01.530567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:53:01.530567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:53:01.530567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:53:01.530567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:53:01.530567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:53:01.530567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:53:01.530567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:53:01.530567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:53:01.530567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:53:01.530567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:53:01.531845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:53:01.531845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:53:01.531845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:53:01.531845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:53:01.531845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:53:01.531845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:53:01.531845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:53:01.531845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:53:01.531845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:53:01.531845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:53:01.531845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:53:01.531845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:53:01.531845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:53:01.531845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:53:01.531845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:53:01.531845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:53:01.531845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:53:01.533353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:53:01.533353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:53:01.533353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:53:01.533353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:53:01.533353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:53:01.533353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:53:01.533353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:53:01.533353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:53:01.536095 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 600/1217] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:53:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:53:01.542307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50f588-53e2-40c8-bd81-0b24f554e119 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:01.681005 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50f588-53e2-40c8-bd81-0b24f554e119 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:01.718402 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:01.730472 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383484f9-60a6-4f76-bd10-9ea47e846958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:01.731080 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383484f9-60a6-4f76-bd10-9ea47e846958 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:01.731356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383484f9-60a6-4f76-bd10-9ea47e846958 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:01.731622 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383484f9-60a6-4f76-bd10-9ea47e846958 None admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:01.732359 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383484f9-60a6-4f76-bd10-9ea47e846958 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:01.732690 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383484f9-60a6-4f76-bd10-9ea47e846958 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:01.734111 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383484f9-60a6-4f76-bd10-9ea47e846958 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:01.752914 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 618/1218] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:53:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:01.755253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54abf13-8a84-4797-aaea-fd9f00eb39ce None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:01.886880 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54abf13-8a84-4797-aaea-fd9f00eb39ce None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:01.919323 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:01.927014 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0bd442-a2fd-4992-8930-c2de3d59d28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:01.927674 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0bd442-a2fd-4992-8930-c2de3d59d28a None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:01.927921 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0bd442-a2fd-4992-8930-c2de3d59d28a None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:01.928229 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0bd442-a2fd-4992-8930-c2de3d59d28a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:01.929360 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0bd442-a2fd-4992-8930-c2de3d59d28a None admin] RBAC: Adding query filter params (role.id=7ebfd9e58fe549f08cdaec5f906a38a6, scope.project.id=80379e2104644e98a2e85d4e4af1b365, user.id=c61e2c954f9a49abad9f3de17ee39f63) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:01.929679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0bd442-a2fd-4992-8930-c2de3d59d28a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:01.931257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0bd442-a2fd-4992-8930-c2de3d59d28a None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:01.939108 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 601/1219] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:53:01 2022] GET /identity/v3/role_assignments?user.id=c61e2c954f9a49abad9f3de17ee39f63&scope.project.id=80379e2104644e98a2e85d4e4af1b365&role.id=7ebfd9e58fe549f08cdaec5f906a38a6 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:04.083876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18663641-02b0-4d41-9e52-747e2464ad33 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:04.084452 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18663641-02b0-4d41-9e52-747e2464ad33 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:04.084452 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18663641-02b0-4d41-9e52-747e2464ad33 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:04.085819 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 619/1220] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:53:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:53:04.091070 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9cca0-dd82-4c0b-bbdf-a65b0ccfacd7 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:04.091525 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9cca0-dd82-4c0b-bbdf-a65b0ccfacd7 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:04.091616 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9cca0-dd82-4c0b-bbdf-a65b0ccfacd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:04.111814 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-d2c9cca0-dd82-4c0b-bbdf-a65b0ccfacd7 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:53:04.113052 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-d2c9cca0-dd82-4c0b-bbdf-a65b0ccfacd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:53:04.200128 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 602/1221] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:53:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:53:04.225084 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18663641-02b0-4d41-9e52-747e2464ad33 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:04.341440 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18663641-02b0-4d41-9e52-747e2464ad33 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:04.387641 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf4d37-e481-4fcf-a2fc-c3f3f0c4ffb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:04.388282 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf4d37-e481-4fcf-a2fc-c3f3f0c4ffb4 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:04.388561 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf4d37-e481-4fcf-a2fc-c3f3f0c4ffb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:04.388911 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf4d37-e481-4fcf-a2fc-c3f3f0c4ffb4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:04.401024 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf4d37-e481-4fcf-a2fc-c3f3f0c4ffb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:53:04.406160 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-e9bf4d37-e481-4fcf-a2fc-c3f3f0c4ffb4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:53:04.406160 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:04.406160 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:53:04.406160 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:53:04.406160 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:53:04.406160 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:53:04.406160 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:53:04.406160 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:53:04.406160 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:53:04.406160 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:53:04.406160 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:53:04.406160 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:53:04.406160 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:53:04.406160 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:53:04.406160 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:53:04.406160 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:53:04.407771 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:53:04.407771 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:53:04.407771 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:53:04.407771 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:53:04.407771 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:53:04.407771 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:53:04.407771 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:53:04.407771 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:53:04.407771 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:53:04.407771 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:53:04.407771 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:53:04.407771 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:53:04.407771 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:53:04.407771 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:53:04.407771 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:53:04.407771 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:53:04.408658 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:53:04.408658 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:53:04.408658 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:53:04.408658 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:53:04.408658 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:53:04.408658 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:53:04.408658 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:53:04.408658 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:53:04.408658 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:53:04.408658 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:53:04.408658 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:53:04.408658 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:53:04.408658 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 620/1222] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Aug 5 19:53:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:53:04.414620 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9cca0-dd82-4c0b-bbdf-a65b0ccfacd7 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:04.541388 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9cca0-dd82-4c0b-bbdf-a65b0ccfacd7 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:04.567184 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:04.581037 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21bc62-9b18-4ead-b8cd-ad1e3496cc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:04.581658 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21bc62-9b18-4ead-b8cd-ad1e3496cc6b None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:04.581939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21bc62-9b18-4ead-b8cd-ad1e3496cc6b None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:04.582214 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21bc62-9b18-4ead-b8cd-ad1e3496cc6b None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:04.582933 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa21bc62-9b18-4ead-b8cd-ad1e3496cc6b None admin] RBAC: Adding query filter params (name=service) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:04.583264 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa21bc62-9b18-4ead-b8cd-ad1e3496cc6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:04.584657 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa21bc62-9b18-4ead-b8cd-ad1e3496cc6b None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:04.598926 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 603/1223] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Aug 5 19:53:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:04.600045 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf4d37-e481-4fcf-a2fc-c3f3f0c4ffb4 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:04.654121 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:04.718118 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf4d37-e481-4fcf-a2fc-c3f3f0c4ffb4 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:04.760906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139bc53-a5e1-4fad-9a62-d930df5567f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:04.761550 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139bc53-a5e1-4fad-9a62-d930df5567f8 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:04.761850 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139bc53-a5e1-4fad-9a62-d930df5567f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:04.762107 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139bc53-a5e1-4fad-9a62-d930df5567f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:04.774919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2139bc53-a5e1-4fad-9a62-d930df5567f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:04.776365 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2139bc53-a5e1-4fad-9a62-d930df5567f8 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:04.791432 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-2139bc53-a5e1-4fad-9a62-d930df5567f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:53:04.791432 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:04.791432 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:53:04.791432 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:53:04.791432 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:53:04.791432 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:53:04.791432 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:53:04.791432 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:53:04.791432 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:53:04.791432 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:53:04.791432 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:53:04.791432 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:53:04.791432 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:53:04.791432 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:53:04.791432 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:53:04.792489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:53:04.792489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:53:04.792489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:53:04.792489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:53:04.792489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:53:04.792489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:53:04.792489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:53:04.792489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:53:04.792489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:53:04.792489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:53:04.792489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:53:04.792489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:53:04.792489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:53:04.792489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:53:04.792489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:53:04.792489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:53:04.792489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:53:04.793822 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:53:04.793822 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:53:04.793822 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:53:04.793822 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:53:04.793822 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 621/1224] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:53:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:53:04.799586 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21bc62-9b18-4ead-b8cd-ad1e3496cc6b None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:04.926634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21bc62-9b18-4ead-b8cd-ad1e3496cc6b None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:04.970135 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e2fb3-e451-43b0-b3da-6dc35d4ca9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:04.970921 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e2fb3-e451-43b0-b3da-6dc35d4ca9ac None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:04.971204 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e2fb3-e451-43b0-b3da-6dc35d4ca9ac None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:04.971473 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e2fb3-e451-43b0-b3da-6dc35d4ca9ac None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:04.972164 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e2fb3-e451-43b0-b3da-6dc35d4ca9ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:04.972467 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e2fb3-e451-43b0-b3da-6dc35d4ca9ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:04.973886 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e2fb3-e451-43b0-b3da-6dc35d4ca9ac None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:04.987081 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 604/1225] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:53:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:04.992851 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139bc53-a5e1-4fad-9a62-d930df5567f8 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:05.102494 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:05.109548 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139bc53-a5e1-4fad-9a62-d930df5567f8 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:05.148606 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ed921-8ad6-4136-8627-4362f7845d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:05.149295 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ed921-8ad6-4136-8627-4362f7845d05 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:05.149579 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ed921-8ad6-4136-8627-4362f7845d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:05.149854 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ed921-8ad6-4136-8627-4362f7845d05 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:05.158614 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ed921-8ad6-4136-8627-4362f7845d05 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:05.160064 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ed921-8ad6-4136-8627-4362f7845d05 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:05.171875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-3f4ed921-8ad6-4136-8627-4362f7845d05 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 19:53:05.171875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:05.171875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:53:05.171875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:53:05.171875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:53:05.171875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:53:05.171875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:53:05.171875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:53:05.171875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:53:05.171875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:53:05.171875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:53:05.171875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:53:05.171875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:53:05.171875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:53:05.171875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:53:05.171875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:53:05.172897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:53:05.172897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:53:05.172897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:53:05.172897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:53:05.172897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:53:05.172897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:53:05.172897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:53:05.172897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:53:05.172897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:53:05.172897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:53:05.172897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:53:05.172897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:53:05.172897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:53:05.172897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:53:05.172897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:53:05.172897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:53:05.172897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:53:05.172897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:53:05.173855 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:53:05.173855 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:53:05.173855 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:53:05.173855 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:53:05.173855 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:53:05.173855 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:53:05.173855 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:53:05.173855 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:53:05.173855 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 19:53:05.173855 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:53:05.173855 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 622/1226] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:53:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:53:05.179580 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e2fb3-e451-43b0-b3da-6dc35d4ca9ac None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:05.232894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:05.290654 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e2fb3-e451-43b0-b3da-6dc35d4ca9ac None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:05.326626 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8028d33-8a54-4aa7-9fa4-43ab8a31fe2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:05.327285 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8028d33-8a54-4aa7-9fa4-43ab8a31fe2f None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:05.327567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8028d33-8a54-4aa7-9fa4-43ab8a31fe2f None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:05.327837 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8028d33-8a54-4aa7-9fa4-43ab8a31fe2f None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:05.329241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8028d33-8a54-4aa7-9fa4-43ab8a31fe2f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:05.329549 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8028d33-8a54-4aa7-9fa4-43ab8a31fe2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:05.331113 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8028d33-8a54-4aa7-9fa4-43ab8a31fe2f None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:05.351755 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 605/1227] 10.222.0.39 () {56 vars in 1143 bytes} [Fri Aug 5 19:53:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:05.357539 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ed921-8ad6-4136-8627-4362f7845d05 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:05.475123 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ed921-8ad6-4136-8627-4362f7845d05 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:05.524133 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93a814-ad0a-4ff8-a8b0-d77a1089fbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:05.524868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93a814-ad0a-4ff8-a8b0-d77a1089fbb7 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:05.525155 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93a814-ad0a-4ff8-a8b0-d77a1089fbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:05.525432 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93a814-ad0a-4ff8-a8b0-d77a1089fbb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:05.536208 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e93a814-ad0a-4ff8-a8b0-d77a1089fbb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:05.537685 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e93a814-ad0a-4ff8-a8b0-d77a1089fbb7 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:05.556520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-4e93a814-ad0a-4ff8-a8b0-d77a1089fbb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:53:05.556520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:05.556520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:53:05.556520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:53:05.556520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:53:05.556520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:53:05.556520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:53:05.556520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:53:05.556520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:53:05.556520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:53:05.556520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:53:05.556520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:53:05.556520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:53:05.556520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:53:05.556520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:53:05.557648 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:53:05.557648 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:53:05.557648 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:53:05.557648 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:53:05.557648 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:53:05.557648 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:53:05.557648 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:53:05.557648 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:53:05.557648 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:53:05.557648 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:53:05.557648 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:53:05.557648 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:53:05.557648 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:53:05.557648 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:53:05.557648 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:53:05.557648 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:53:05.557648 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:53:05.558552 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:53:05.558552 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:53:05.558552 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:53:05.558552 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:53:05.558552 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 623/1228] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:53:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:53:05.565214 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8028d33-8a54-4aa7-9fa4-43ab8a31fe2f None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:05.703077 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8028d33-8a54-4aa7-9fa4-43ab8a31fe2f None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:05.705732 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:05.744738 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d007f5-43fd-4244-812e-fb3c8ba97c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:05.750945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d007f5-43fd-4244-812e-fb3c8ba97c0c None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:05.750945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d007f5-43fd-4244-812e-fb3c8ba97c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:05.750945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d007f5-43fd-4244-812e-fb3c8ba97c0c None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:05.750945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d007f5-43fd-4244-812e-fb3c8ba97c0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:05.750945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d007f5-43fd-4244-812e-fb3c8ba97c0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:05.750945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d007f5-43fd-4244-812e-fb3c8ba97c0c None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:05.765974 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 606/1229] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:53:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:05.771940 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93a814-ad0a-4ff8-a8b0-d77a1089fbb7 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:05.781438 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:05.914281 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93a814-ad0a-4ff8-a8b0-d77a1089fbb7 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:05.959078 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce81bf-1584-4407-b2b9-69e029f96fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:05.959718 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce81bf-1584-4407-b2b9-69e029f96fae None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:05.960018 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce81bf-1584-4407-b2b9-69e029f96fae None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:05.960270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce81bf-1584-4407-b2b9-69e029f96fae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:05.967753 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ce81bf-1584-4407-b2b9-69e029f96fae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:05.970011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ce81bf-1584-4407-b2b9-69e029f96fae None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:05.982114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-e0ce81bf-1584-4407-b2b9-69e029f96fae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:53:05.982114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:05.982114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:53:05.982114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:53:05.982114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:53:05.982114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:53:05.982114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:53:05.982114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:53:05.982114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:53:05.982114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:53:05.982114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:53:05.982114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:53:05.982114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:53:05.982114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:53:05.982114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:53:05.983220 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:53:05.983220 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:53:05.983220 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:53:05.983220 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:53:05.983220 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:53:05.983220 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:53:05.983220 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:53:05.983220 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:53:05.983220 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:53:05.983220 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:53:05.983220 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:53:05.983220 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:53:05.983220 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:53:05.983220 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:53:05.983220 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:53:05.983220 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:53:05.983220 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:53:05.984156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:53:05.984156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:53:05.984156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:53:05.984156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:53:05.984156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:53:05.984156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:53:05.984156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:53:05.984156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:53:05.984156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 624/1230] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:53:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:53:05.990556 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d007f5-43fd-4244-812e-fb3c8ba97c0c None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:06.105946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d007f5-43fd-4244-812e-fb3c8ba97c0c None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:06.151494 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e9749-a636-432e-87b5-dea21f08787d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:06.152205 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e9749-a636-432e-87b5-dea21f08787d None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:06.152461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e9749-a636-432e-87b5-dea21f08787d None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:06.152923 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e9749-a636-432e-87b5-dea21f08787d None admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:06.153808 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e9749-a636-432e-87b5-dea21f08787d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:06.154124 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e9749-a636-432e-87b5-dea21f08787d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:06.155975 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e9749-a636-432e-87b5-dea21f08787d None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:06.170727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 607/1231] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:53:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:06.181415 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce81bf-1584-4407-b2b9-69e029f96fae None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:06.257519 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:06.307766 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce81bf-1584-4407-b2b9-69e029f96fae None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:06.350418 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58cabfb-59a3-43b6-b8ba-83390d99b11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:06.351134 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cabfb-59a3-43b6-b8ba-83390d99b11a None admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:06.351378 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cabfb-59a3-43b6-b8ba-83390d99b11a None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:06.351655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cabfb-59a3-43b6-b8ba-83390d99b11a None admin] PATH_INFO: `/v3/projects/80379e2104644e98a2e85d4e4af1b365/users/c61e2c954f9a49abad9f3de17ee39f63/roles/7ebfd9e58fe549f08cdaec5f906a38a6` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:06.367901 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58cabfb-59a3-43b6-b8ba-83390d99b11a None admin] RBAC: Authorizing `identity:create_grant(project_id=80379e2104644e98a2e85d4e4af1b365, user_id=c61e2c954f9a49abad9f3de17ee39f63, role_id=7ebfd9e58fe549f08cdaec5f906a38a6)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:06.369333 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58cabfb-59a3-43b6-b8ba-83390d99b11a None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:06.382444 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 625/1232] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Aug 5 19:53:06 2022] PUT /identity/v3/projects/80379e2104644e98a2e85d4e4af1b365/users/c61e2c954f9a49abad9f3de17ee39f63/roles/7ebfd9e58fe549f08cdaec5f906a38a6 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:53:08.853444 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d33765-955a-4ecf-8636-9f5f1954d984 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:08.854192 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d33765-955a-4ecf-8636-9f5f1954d984 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:08.854192 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d33765-955a-4ecf-8636-9f5f1954d984 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:08.855344 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 608/1233] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:53:08.861337 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6a33b-a559-4fb2-8510-454e9d6047f5 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:08.861836 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6a33b-a559-4fb2-8510-454e9d6047f5 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:08.861935 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6a33b-a559-4fb2-8510-454e9d6047f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:08.881564 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-b9e6a33b-a559-4fb2-8510-454e9d6047f5 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:53:08.882762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-b9e6a33b-a559-4fb2-8510-454e9d6047f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:53:08.956114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 626/1234] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:53:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:53:08.979739 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d33765-955a-4ecf-8636-9f5f1954d984 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:09.006320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:09.097759 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d33765-955a-4ecf-8636-9f5f1954d984 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:09.138541 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c7fb9-2142-4cfa-9c76-68eb3dbf2094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:09.139180 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c7fb9-2142-4cfa-9c76-68eb3dbf2094 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:09.139917 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c7fb9-2142-4cfa-9c76-68eb3dbf2094 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:09.140193 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c7fb9-2142-4cfa-9c76-68eb3dbf2094 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:09.152208 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6c7fb9-2142-4cfa-9c76-68eb3dbf2094 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:53:09.158569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-0c6c7fb9-2142-4cfa-9c76-68eb3dbf2094 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:53:09.158569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:09.158569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:53:09.158569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:53:09.158569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:53:09.158569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:53:09.158569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:53:09.158569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:53:09.158569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:53:09.158569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:53:09.158569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:53:09.158569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:53:09.158569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:53:09.158569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:53:09.158569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:53:09.158569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:53:09.159690 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:53:09.159690 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:53:09.159690 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:53:09.159690 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:53:09.159690 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:53:09.159690 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:53:09.159690 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:53:09.159690 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:53:09.159690 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:53:09.159690 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:53:09.159690 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:53:09.159690 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:53:09.159690 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:53:09.159690 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:53:09.159690 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:53:09.159690 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:53:09.160686 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:53:09.160686 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:53:09.160686 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:53:09.160686 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:53:09.160686 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:53:09.160686 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:53:09.160686 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:53:09.160686 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:53:09.160686 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:53:09.160686 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:53:09.160686 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:53:09.160686 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:53:09.160686 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 609/1235] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Aug 5 19:53:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:53:09.165925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6a33b-a559-4fb2-8510-454e9d6047f5 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:09.219845 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:09.284339 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6a33b-a559-4fb2-8510-454e9d6047f5 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:09.318583 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed69e5-e22d-4273-a3b6-23b2838d33bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:09.319192 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed69e5-e22d-4273-a3b6-23b2838d33bc None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:09.319468 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed69e5-e22d-4273-a3b6-23b2838d33bc None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:09.319740 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed69e5-e22d-4273-a3b6-23b2838d33bc None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:09.320463 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed69e5-e22d-4273-a3b6-23b2838d33bc None admin] RBAC: Adding query filter params (name=service) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:09.320901 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed69e5-e22d-4273-a3b6-23b2838d33bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:09.322526 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed69e5-e22d-4273-a3b6-23b2838d33bc None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:09.332047 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 627/1236] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Aug 5 19:53:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:09.337814 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c7fb9-2142-4cfa-9c76-68eb3dbf2094 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:09.440924 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:09.462084 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c7fb9-2142-4cfa-9c76-68eb3dbf2094 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:09.501648 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8ff5f-ed2f-4bf0-91d5-8d4eb07b8cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:09.502507 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8ff5f-ed2f-4bf0-91d5-8d4eb07b8cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:09.502975 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8ff5f-ed2f-4bf0-91d5-8d4eb07b8cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:09.503377 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8ff5f-ed2f-4bf0-91d5-8d4eb07b8cb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:09.514058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8ff5f-ed2f-4bf0-91d5-8d4eb07b8cb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:09.515543 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8ff5f-ed2f-4bf0-91d5-8d4eb07b8cb3 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:09.531733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-0fa8ff5f-ed2f-4bf0-91d5-8d4eb07b8cb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:53:09.531733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:09.531733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:53:09.531733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:53:09.531733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:53:09.531733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:53:09.531733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:53:09.531733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:53:09.531733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:53:09.531733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:53:09.531733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:53:09.531733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:53:09.531733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:53:09.531733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:53:09.531733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:53:09.537639 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:53:09.537639 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:53:09.537639 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:53:09.537639 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:53:09.537639 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:53:09.537639 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:53:09.537639 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:53:09.537639 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:53:09.537639 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:53:09.537639 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:53:09.537639 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:53:09.537639 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:53:09.537639 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:53:09.537639 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:53:09.537639 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:53:09.537639 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:53:09.537639 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:53:09.538773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:53:09.538773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:53:09.538773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:53:09.538773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:53:09.538773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 610/1237] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:53:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:53:09.540434 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed69e5-e22d-4273-a3b6-23b2838d33bc None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:09.664741 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed69e5-e22d-4273-a3b6-23b2838d33bc None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:09.675866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:09.705737 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd2bdf4-18c9-485a-ae92-3cd58b16f4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:09.706347 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2bdf4-18c9-485a-ae92-3cd58b16f4ee None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:09.706655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2bdf4-18c9-485a-ae92-3cd58b16f4ee None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:09.706924 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2bdf4-18c9-485a-ae92-3cd58b16f4ee None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:09.707606 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd2bdf4-18c9-485a-ae92-3cd58b16f4ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:09.707907 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd2bdf4-18c9-485a-ae92-3cd58b16f4ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:09.709291 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd2bdf4-18c9-485a-ae92-3cd58b16f4ee None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:09.728476 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 628/1238] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:53:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:09.735570 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8ff5f-ed2f-4bf0-91d5-8d4eb07b8cb3 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:09.870921 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8ff5f-ed2f-4bf0-91d5-8d4eb07b8cb3 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:09.907869 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b406f-42f2-4669-a7ed-fe8df69ff7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:09.908518 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b406f-42f2-4669-a7ed-fe8df69ff7d2 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:09.908828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b406f-42f2-4669-a7ed-fe8df69ff7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:09.909103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b406f-42f2-4669-a7ed-fe8df69ff7d2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:09.912066 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:09.918743 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355b406f-42f2-4669-a7ed-fe8df69ff7d2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:09.920519 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355b406f-42f2-4669-a7ed-fe8df69ff7d2 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:09.935887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-355b406f-42f2-4669-a7ed-fe8df69ff7d2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 19:53:09.935887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:09.935887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:53:09.935887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:53:09.935887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:53:09.935887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:53:09.935887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:53:09.935887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:53:09.935887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:53:09.935887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:53:09.935887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:53:09.935887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:53:09.935887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:53:09.935887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:53:09.935887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:53:09.935887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:53:09.936979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:53:09.936979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:53:09.936979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:53:09.936979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:53:09.936979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:53:09.936979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:53:09.936979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:53:09.936979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:53:09.936979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:53:09.936979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:53:09.936979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:53:09.936979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:53:09.936979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:53:09.936979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:53:09.936979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:53:09.936979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:53:09.936979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:53:09.936979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:53:09.938560 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:53:09.938560 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:53:09.938560 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:53:09.938560 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:53:09.938560 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:53:09.938560 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:53:09.938560 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:53:09.938560 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:53:09.938560 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 19:53:09.938560 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:53:09.938560 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 611/1239] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:53:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:53:09.943309 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd2bdf4-18c9-485a-ae92-3cd58b16f4ee None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:10.057467 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd2bdf4-18c9-485a-ae92-3cd58b16f4ee None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:10.095069 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f0ab2-8bfc-4cd4-92c0-a2f2acb2c235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:10.095818 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f0ab2-8bfc-4cd4-92c0-a2f2acb2c235 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:10.096094 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f0ab2-8bfc-4cd4-92c0-a2f2acb2c235 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:10.096426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f0ab2-8bfc-4cd4-92c0-a2f2acb2c235 None admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:10.097316 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553f0ab2-8bfc-4cd4-92c0-a2f2acb2c235 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:10.097623 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553f0ab2-8bfc-4cd4-92c0-a2f2acb2c235 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:10.099099 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553f0ab2-8bfc-4cd4-92c0-a2f2acb2c235 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:10.119004 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 629/1240] 10.222.0.39 () {56 vars in 1143 bytes} [Fri Aug 5 19:53:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:10.125051 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b406f-42f2-4669-a7ed-fe8df69ff7d2 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:10.246360 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b406f-42f2-4669-a7ed-fe8df69ff7d2 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:10.286716 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb646233-5cbb-4649-935d-8aaf59eb97cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:10.287341 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb646233-5cbb-4649-935d-8aaf59eb97cf None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:10.287682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb646233-5cbb-4649-935d-8aaf59eb97cf None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:10.287997 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb646233-5cbb-4649-935d-8aaf59eb97cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:10.364118 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:10.371023 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb646233-5cbb-4649-935d-8aaf59eb97cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:10.372620 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb646233-5cbb-4649-935d-8aaf59eb97cf None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:10.494109 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 612/1241] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:53:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:53:10.500962 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f0ab2-8bfc-4cd4-92c0-a2f2acb2c235 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:10.537736 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:10.634241 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f0ab2-8bfc-4cd4-92c0-a2f2acb2c235 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:10.678791 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13a658-9cbd-48f3-a488-61ade6abcc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:10.679443 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13a658-9cbd-48f3-a488-61ade6abcc19 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:10.679721 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13a658-9cbd-48f3-a488-61ade6abcc19 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:10.679991 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13a658-9cbd-48f3-a488-61ade6abcc19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:10.691486 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f13a658-9cbd-48f3-a488-61ade6abcc19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:10.693155 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f13a658-9cbd-48f3-a488-61ade6abcc19 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:10.709862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-3f13a658-9cbd-48f3-a488-61ade6abcc19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:53:10.709862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:10.709862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:53:10.709862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:53:10.709862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:53:10.709862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:53:10.709862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:53:10.709862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:53:10.709862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:53:10.709862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:53:10.709862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:53:10.709862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:53:10.709862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:53:10.709862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:53:10.709862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:53:10.710994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:53:10.710994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:53:10.710994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:53:10.710994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:53:10.710994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:53:10.710994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:53:10.710994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:53:10.710994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:53:10.710994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:53:10.710994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:53:10.710994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:53:10.710994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:53:10.710994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:53:10.710994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:53:10.710994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:53:10.710994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:53:10.710994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:53:10.712049 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:53:10.712049 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:53:10.712049 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:53:10.712049 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:53:10.712049 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 630/1242] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:53:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:53:10.717284 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb646233-5cbb-4649-935d-8aaf59eb97cf None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:10.828953 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:10.835360 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb646233-5cbb-4649-935d-8aaf59eb97cf None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:10.875736 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d6640-1994-4b0f-8211-97bf296614b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:10.876464 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d6640-1994-4b0f-8211-97bf296614b2 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:10.876707 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d6640-1994-4b0f-8211-97bf296614b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:10.877013 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d6640-1994-4b0f-8211-97bf296614b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:10.877704 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d6640-1994-4b0f-8211-97bf296614b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:10.878064 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d6640-1994-4b0f-8211-97bf296614b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:10.879418 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d6640-1994-4b0f-8211-97bf296614b2 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:10.892259 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 613/1243] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:53:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:53:10.898394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13a658-9cbd-48f3-a488-61ade6abcc19 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:10.990601 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:11.014813 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13a658-9cbd-48f3-a488-61ade6abcc19 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:11.053370 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ce782-5376-42d5-849c-b75c5337a191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:11.054086 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ce782-5376-42d5-849c-b75c5337a191 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:11.054391 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ce782-5376-42d5-849c-b75c5337a191 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:11.054648 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ce782-5376-42d5-849c-b75c5337a191 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:11.061879 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70ce782-5376-42d5-849c-b75c5337a191 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:11.064010 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70ce782-5376-42d5-849c-b75c5337a191 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:11.076404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-c70ce782-5376-42d5-849c-b75c5337a191 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:53:11.076404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:11.076404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:53:11.076404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:53:11.076404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:53:11.076404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:53:11.076404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:53:11.076404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:53:11.076404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:53:11.076404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:53:11.076404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:53:11.076404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:53:11.076404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:53:11.076404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:53:11.076404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:53:11.077574 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:53:11.077574 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:53:11.077574 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:53:11.077574 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:53:11.077574 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:53:11.077574 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:53:11.077574 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:53:11.077574 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:53:11.077574 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:53:11.077574 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:53:11.077574 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:53:11.077574 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:53:11.077574 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:53:11.077574 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:53:11.077574 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:53:11.077574 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:53:11.077574 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:53:11.078606 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:53:11.078606 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:53:11.078606 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:53:11.078606 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:53:11.078606 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:53:11.078606 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:53:11.078606 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:53:11.078606 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:53:11.078606 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 631/1244] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:53:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:53:11.085643 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d6640-1994-4b0f-8211-97bf296614b2 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:11.197424 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d6640-1994-4b0f-8211-97bf296614b2 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:11.236102 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2f3129-1a95-496d-ac79-a53b5ba88c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:11.236759 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f3129-1a95-496d-ac79-a53b5ba88c38 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:11.237366 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f3129-1a95-496d-ac79-a53b5ba88c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:11.237366 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f3129-1a95-496d-ac79-a53b5ba88c38 None admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:11.238045 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2f3129-1a95-496d-ac79-a53b5ba88c38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:11.238330 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2f3129-1a95-496d-ac79-a53b5ba88c38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:11.239749 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2f3129-1a95-496d-ac79-a53b5ba88c38 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:11.256263 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 614/1245] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:53:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:11.262634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ce782-5376-42d5-849c-b75c5337a191 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:11.374195 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ce782-5376-42d5-849c-b75c5337a191 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:11.411042 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdfe2ba-c58c-439a-8a52-1eac9af73633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:11.411588 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfe2ba-c58c-439a-8a52-1eac9af73633 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:11.411873 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfe2ba-c58c-439a-8a52-1eac9af73633 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:11.412146 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfe2ba-c58c-439a-8a52-1eac9af73633 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:11.412974 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdfe2ba-c58c-439a-8a52-1eac9af73633 None admin] RBAC: Adding query filter params (role.id=7ebfd9e58fe549f08cdaec5f906a38a6, scope.project.id=80379e2104644e98a2e85d4e4af1b365, user.id=c61e2c954f9a49abad9f3de17ee39f63) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:11.413288 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdfe2ba-c58c-439a-8a52-1eac9af73633 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:11.414723 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdfe2ba-c58c-439a-8a52-1eac9af73633 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:11.421031 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 632/1246] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:53:11 2022] GET /identity/v3/role_assignments?user.id=c61e2c954f9a49abad9f3de17ee39f63&scope.project.id=80379e2104644e98a2e85d4e4af1b365&role.id=7ebfd9e58fe549f08cdaec5f906a38a6 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:53:13.646311 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de7a0e-4440-404e-a0b1-cd510674570c None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:13.647026 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de7a0e-4440-404e-a0b1-cd510674570c None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:13.647026 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de7a0e-4440-404e-a0b1-cd510674570c None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:13.648557 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 615/1247] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:53:13.656378 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5804ba9-8956-46d6-9efd-f4c7461bddc9 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:13.657196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5804ba9-8956-46d6-9efd-f4c7461bddc9 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:13.657196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5804ba9-8956-46d6-9efd-f4c7461bddc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:13.664301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:13.686473 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-d5804ba9-8956-46d6-9efd-f4c7461bddc9 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:53:13.687839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-d5804ba9-8956-46d6-9efd-f4c7461bddc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:53:13.782849 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 633/1248] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:53:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:53:13.806770 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de7a0e-4440-404e-a0b1-cd510674570c None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:13.829606 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:13.943913 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de7a0e-4440-404e-a0b1-cd510674570c None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:13.994546 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27843857-6d69-4c17-aa06-3da7511a227a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:13.995228 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27843857-6d69-4c17-aa06-3da7511a227a None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:13.995511 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27843857-6d69-4c17-aa06-3da7511a227a None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:13.995791 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27843857-6d69-4c17-aa06-3da7511a227a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:14.008189 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27843857-6d69-4c17-aa06-3da7511a227a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:53:14.013227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-27843857-6d69-4c17-aa06-3da7511a227a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:53:14.013227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:14.013227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:53:14.013227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:53:14.013227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:53:14.013227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:53:14.013227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:53:14.013227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:53:14.013227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:53:14.013227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:53:14.013227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:53:14.013227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:53:14.013227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:53:14.013227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:53:14.013227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:53:14.013227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:53:14.014232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:53:14.014232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:53:14.014232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:53:14.014232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:53:14.014232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:53:14.014232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:53:14.014232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:53:14.014232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:53:14.014232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:53:14.014232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:53:14.014232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:53:14.014232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:53:14.014232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:53:14.014232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:53:14.014232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:53:14.014232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:53:14.015802 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:53:14.015802 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:53:14.015802 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:53:14.015802 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:53:14.015802 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:53:14.015802 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:53:14.015802 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:53:14.015802 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:53:14.015802 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:53:14.015802 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:53:14.015802 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:53:14.015802 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:53:14.015802 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 616/1249] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:53:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:53:14.021988 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5804ba9-8956-46d6-9efd-f4c7461bddc9 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:14.143600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5804ba9-8956-46d6-9efd-f4c7461bddc9 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:14.188917 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:14.195584 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a1dfa-2a0a-4246-949f-210536b23b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:14.196280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a1dfa-2a0a-4246-949f-210536b23b41 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:14.196938 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a1dfa-2a0a-4246-949f-210536b23b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:14.197429 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a1dfa-2a0a-4246-949f-210536b23b41 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:14.198507 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178a1dfa-2a0a-4246-949f-210536b23b41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:14.199204 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178a1dfa-2a0a-4246-949f-210536b23b41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:14.200999 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178a1dfa-2a0a-4246-949f-210536b23b41 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:14.215415 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 634/1250] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:53:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:14.222055 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27843857-6d69-4c17-aa06-3da7511a227a None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:14.332422 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:14.371752 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27843857-6d69-4c17-aa06-3da7511a227a None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:14.415624 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7e554-fe6f-4bb8-be40-68aaeb1c9624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:14.417042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7e554-fe6f-4bb8-be40-68aaeb1c9624 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:14.417280 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7e554-fe6f-4bb8-be40-68aaeb1c9624 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:14.417636 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7e554-fe6f-4bb8-be40-68aaeb1c9624 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:14.432442 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7e554-fe6f-4bb8-be40-68aaeb1c9624 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:14.434302 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7e554-fe6f-4bb8-be40-68aaeb1c9624 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:14.455205 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-c4b7e554-fe6f-4bb8-be40-68aaeb1c9624 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:53:14.455205 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:14.455205 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:53:14.455205 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:53:14.455205 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:53:14.455205 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:53:14.455205 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:53:14.455205 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:53:14.455205 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:53:14.455205 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:53:14.455205 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:53:14.455205 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:53:14.455205 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:53:14.455205 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:53:14.455205 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:53:14.456359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:53:14.456359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:53:14.456359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:53:14.456359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:53:14.456359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:53:14.456359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:53:14.456359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:53:14.456359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:53:14.456359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:53:14.456359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:53:14.456359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:53:14.456359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:53:14.456359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:53:14.456359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:53:14.456359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:53:14.456359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:53:14.456359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:53:14.457498 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:53:14.457498 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:53:14.457498 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:53:14.457498 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:53:14.457498 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 617/1251] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:53:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:53:14.469468 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a1dfa-2a0a-4246-949f-210536b23b41 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:14.594274 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a1dfa-2a0a-4246-949f-210536b23b41 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:14.654089 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca258b3d-51dd-4fd5-9109-b9ab0edcffc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:14.654720 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca258b3d-51dd-4fd5-9109-b9ab0edcffc4 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:14.654999 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca258b3d-51dd-4fd5-9109-b9ab0edcffc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:14.655274 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca258b3d-51dd-4fd5-9109-b9ab0edcffc4 None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:14.656773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca258b3d-51dd-4fd5-9109-b9ab0edcffc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:14.657640 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca258b3d-51dd-4fd5-9109-b9ab0edcffc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:14.658517 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca258b3d-51dd-4fd5-9109-b9ab0edcffc4 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:14.672191 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 635/1252] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:53:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:14.679343 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7e554-fe6f-4bb8-be40-68aaeb1c9624 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:14.811046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7e554-fe6f-4bb8-be40-68aaeb1c9624 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:14.838856 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:14.848251 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10dd8db-1f00-4cb2-975d-750207b07649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:14.848920 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10dd8db-1f00-4cb2-975d-750207b07649 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:14.849201 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10dd8db-1f00-4cb2-975d-750207b07649 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:14.849536 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10dd8db-1f00-4cb2-975d-750207b07649 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:14.862923 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10dd8db-1f00-4cb2-975d-750207b07649 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:14.864375 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10dd8db-1f00-4cb2-975d-750207b07649 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:14.877287 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-a10dd8db-1f00-4cb2-975d-750207b07649 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 19:53:14.877287 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:14.877287 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:53:14.877287 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:53:14.877287 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:53:14.877287 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:53:14.877287 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:53:14.877287 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:53:14.877287 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:53:14.877287 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:53:14.877287 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:53:14.877287 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:53:14.877287 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:53:14.877287 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:53:14.877287 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:53:14.877287 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:53:14.878615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:53:14.878615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:53:14.878615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:53:14.878615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:53:14.878615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:53:14.878615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:53:14.878615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:53:14.878615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:53:14.878615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:53:14.878615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:53:14.878615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:53:14.878615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:53:14.878615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:53:14.878615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:53:14.878615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:53:14.878615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:53:14.878615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:53:14.878615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:53:14.879817 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:53:14.879817 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:53:14.879817 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:53:14.879817 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:53:14.879817 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:53:14.879817 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:53:14.879817 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:53:14.879817 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:53:14.879817 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 19:53:14.879817 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:53:14.879817 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 618/1253] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:53:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:53:14.884447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca258b3d-51dd-4fd5-9109-b9ab0edcffc4 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:14.947938 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:15.005764 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca258b3d-51dd-4fd5-9109-b9ab0edcffc4 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:15.045271 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192346e7-f311-4df4-b6d3-433e861e9a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:15.046043 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192346e7-f311-4df4-b6d3-433e861e9a4a None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:15.046961 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192346e7-f311-4df4-b6d3-433e861e9a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:15.047329 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192346e7-f311-4df4-b6d3-433e861e9a4a None admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:15.048144 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192346e7-f311-4df4-b6d3-433e861e9a4a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:15.048456 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192346e7-f311-4df4-b6d3-433e861e9a4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:15.049956 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192346e7-f311-4df4-b6d3-433e861e9a4a None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:15.070550 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 636/1254] 10.222.0.39 () {56 vars in 1143 bytes} [Fri Aug 5 19:53:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:15.076305 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10dd8db-1f00-4cb2-975d-750207b07649 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:15.194121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10dd8db-1f00-4cb2-975d-750207b07649 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:15.235530 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0117cfbb-4c52-4e04-9e64-67d6a726078a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:15.236150 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0117cfbb-4c52-4e04-9e64-67d6a726078a None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:15.236422 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0117cfbb-4c52-4e04-9e64-67d6a726078a None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:15.236733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0117cfbb-4c52-4e04-9e64-67d6a726078a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:15.310127 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:15.325874 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0117cfbb-4c52-4e04-9e64-67d6a726078a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:15.327551 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0117cfbb-4c52-4e04-9e64-67d6a726078a None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:15.445412 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 619/1255] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:53:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:53:15.451327 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192346e7-f311-4df4-b6d3-433e861e9a4a None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:15.566770 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192346e7-f311-4df4-b6d3-433e861e9a4a None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:15.582874 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:15.607584 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c8aad-6eed-4fd3-a124-86c979b83223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:15.608226 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c8aad-6eed-4fd3-a124-86c979b83223 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:15.608701 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c8aad-6eed-4fd3-a124-86c979b83223 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:15.609063 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c8aad-6eed-4fd3-a124-86c979b83223 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:15.623645 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c8aad-6eed-4fd3-a124-86c979b83223 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:15.625229 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c8aad-6eed-4fd3-a124-86c979b83223 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:15.643936 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-679c8aad-6eed-4fd3-a124-86c979b83223 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:53:15.643936 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:15.643936 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:53:15.643936 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:53:15.643936 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:53:15.643936 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:53:15.643936 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:53:15.643936 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:53:15.643936 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:53:15.643936 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:53:15.643936 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:53:15.643936 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:53:15.643936 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:53:15.643936 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:53:15.643936 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:53:15.646675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:53:15.646675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:53:15.646675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:53:15.646675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:53:15.646675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:53:15.646675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:53:15.646675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:53:15.646675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:53:15.646675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:53:15.646675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:53:15.646675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:53:15.646675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:53:15.646675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:53:15.646675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:53:15.646675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:53:15.646675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:53:15.646675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:53:15.648846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:53:15.648846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:53:15.648846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:53:15.648846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:53:15.648846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 637/1256] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:53:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:53:15.651665 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0117cfbb-4c52-4e04-9e64-67d6a726078a None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:15.760173 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:15.775858 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0117cfbb-4c52-4e04-9e64-67d6a726078a None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:15.819002 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f72e3-c9be-4508-a18b-80fbd54ad460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:15.819663 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f72e3-c9be-4508-a18b-80fbd54ad460 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:15.820034 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f72e3-c9be-4508-a18b-80fbd54ad460 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:15.820344 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f72e3-c9be-4508-a18b-80fbd54ad460 None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:15.821228 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f72e3-c9be-4508-a18b-80fbd54ad460 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:15.821639 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f72e3-c9be-4508-a18b-80fbd54ad460 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:15.823526 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f72e3-c9be-4508-a18b-80fbd54ad460 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:15.839459 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 620/1257] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:53:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:15.846014 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c8aad-6eed-4fd3-a124-86c979b83223 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:15.957821 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c8aad-6eed-4fd3-a124-86c979b83223 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:15.997073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30003229-f24f-4022-a860-0163468fbaeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:15.998070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30003229-f24f-4022-a860-0163468fbaeb None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:15.998434 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30003229-f24f-4022-a860-0163468fbaeb None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:15.998787 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30003229-f24f-4022-a860-0163468fbaeb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:16.007525 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30003229-f24f-4022-a860-0163468fbaeb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:16.009281 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30003229-f24f-4022-a860-0163468fbaeb None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:16.027072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-30003229-f24f-4022-a860-0163468fbaeb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:53:16.027072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:16.027072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:53:16.027072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:53:16.027072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:53:16.027072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:53:16.027072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:53:16.027072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:53:16.027072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:53:16.027072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:53:16.027072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:53:16.027072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:53:16.027072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:53:16.027072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:53:16.027072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:53:16.028198 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:53:16.028198 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:53:16.028198 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:53:16.028198 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:53:16.028198 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:53:16.028198 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:53:16.028198 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:53:16.028198 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:53:16.028198 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:53:16.028198 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:53:16.028198 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:53:16.028198 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:53:16.028198 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:53:16.028198 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:53:16.028198 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:53:16.028198 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:53:16.028198 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:53:16.034467 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:53:16.034467 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:53:16.034467 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:53:16.034467 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:53:16.034467 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:53:16.034467 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:53:16.034467 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:53:16.034467 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:53:16.034467 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 638/1258] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:53:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:53:16.037070 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f72e3-c9be-4508-a18b-80fbd54ad460 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:16.160183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f72e3-c9be-4508-a18b-80fbd54ad460 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:16.201538 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f96d5c-e0cb-4ad8-b6cd-c02de1094494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:16.202201 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f96d5c-e0cb-4ad8-b6cd-c02de1094494 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:16.202641 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f96d5c-e0cb-4ad8-b6cd-c02de1094494 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:16.203001 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f96d5c-e0cb-4ad8-b6cd-c02de1094494 None admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:16.203852 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f96d5c-e0cb-4ad8-b6cd-c02de1094494 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:16.204164 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f96d5c-e0cb-4ad8-b6cd-c02de1094494 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:16.205764 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f96d5c-e0cb-4ad8-b6cd-c02de1094494 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:16.227375 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 621/1259] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:53:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:16.235882 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30003229-f24f-4022-a860-0163468fbaeb None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:16.271130 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:16.370413 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30003229-f24f-4022-a860-0163468fbaeb None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:16.409737 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c300b1d-fc16-458a-9031-fd1a93ddc3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:16.410269 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c300b1d-fc16-458a-9031-fd1a93ddc3fc None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:16.410542 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c300b1d-fc16-458a-9031-fd1a93ddc3fc None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:16.410808 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c300b1d-fc16-458a-9031-fd1a93ddc3fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:16.411598 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c300b1d-fc16-458a-9031-fd1a93ddc3fc None admin] RBAC: Adding query filter params (role.id=89919f784ad144529b07c8ab92ee7d48, scope.project.id=80379e2104644e98a2e85d4e4af1b365, user.id=c61e2c954f9a49abad9f3de17ee39f63) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:16.411930 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c300b1d-fc16-458a-9031-fd1a93ddc3fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:16.413398 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c300b1d-fc16-458a-9031-fd1a93ddc3fc None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:16.419132 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 639/1260] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:53:16 2022] GET /identity/v3/role_assignments?user.id=c61e2c954f9a49abad9f3de17ee39f63&scope.project.id=80379e2104644e98a2e85d4e4af1b365&role.id=89919f784ad144529b07c8ab92ee7d48 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:18.549827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca811e86-4dbe-4759-a7a1-a4c45254903e None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:18.550337 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca811e86-4dbe-4759-a7a1-a4c45254903e None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:18.550337 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca811e86-4dbe-4759-a7a1-a4c45254903e None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:18.551641 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 622/1261] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:53:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:53:18.557484 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4c41a-3898-4c90-bc35-86dcaf02b982 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:18.557932 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4c41a-3898-4c90-bc35-86dcaf02b982 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:18.558025 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4c41a-3898-4c90-bc35-86dcaf02b982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:18.576446 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-34d4c41a-3898-4c90-bc35-86dcaf02b982 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:53:18.578355 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-34d4c41a-3898-4c90-bc35-86dcaf02b982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:53:18.660714 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 640/1262] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:53:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:53:18.684257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca811e86-4dbe-4759-a7a1-a4c45254903e None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:18.695124 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:18.788783 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca811e86-4dbe-4759-a7a1-a4c45254903e None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:18.824245 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af31b8a-8af2-48bb-8ed7-1d4ca0b321a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:18.824834 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af31b8a-8af2-48bb-8ed7-1d4ca0b321a4 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:18.825111 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af31b8a-8af2-48bb-8ed7-1d4ca0b321a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:18.825382 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af31b8a-8af2-48bb-8ed7-1d4ca0b321a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:18.836348 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af31b8a-8af2-48bb-8ed7-1d4ca0b321a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:53:18.841177 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-2af31b8a-8af2-48bb-8ed7-1d4ca0b321a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:53:18.841177 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:18.841177 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:53:18.841177 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:53:18.841177 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:53:18.841177 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:53:18.841177 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:53:18.841177 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:53:18.841177 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:53:18.841177 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:53:18.841177 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:53:18.841177 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:53:18.841177 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:53:18.841177 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:53:18.841177 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:53:18.841177 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:53:18.842363 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:53:18.842363 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:53:18.842363 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:53:18.842363 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:53:18.842363 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:53:18.842363 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:53:18.842363 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:53:18.842363 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:53:18.842363 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:53:18.842363 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:53:18.842363 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:53:18.842363 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:53:18.842363 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:53:18.842363 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:53:18.842363 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:53:18.842363 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:53:18.843247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:53:18.843247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:53:18.843247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:53:18.843247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:53:18.843247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:53:18.843247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:53:18.843247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:53:18.843247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:53:18.843247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:53:18.843247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:53:18.843247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:53:18.843247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:53:18.843247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 623/1263] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:53:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:53:18.849077 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4c41a-3898-4c90-bc35-86dcaf02b982 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:18.869428 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:18.958287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4c41a-3898-4c90-bc35-86dcaf02b982 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:18.993078 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f428905-5616-4709-b5e4-d42f7a480513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:18.993693 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f428905-5616-4709-b5e4-d42f7a480513 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:18.993974 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f428905-5616-4709-b5e4-d42f7a480513 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:18.994247 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f428905-5616-4709-b5e4-d42f7a480513 None admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:18.994963 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f428905-5616-4709-b5e4-d42f7a480513 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:18.995269 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f428905-5616-4709-b5e4-d42f7a480513 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:18.996652 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f428905-5616-4709-b5e4-d42f7a480513 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:19.004989 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 641/1264] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:53:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:19.011126 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af31b8a-8af2-48bb-8ed7-1d4ca0b321a4 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:19.080979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:19.115100 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af31b8a-8af2-48bb-8ed7-1d4ca0b321a4 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:19.149565 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f5f03-7906-4bb6-8ed8-9474331cdc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:19.150245 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f5f03-7906-4bb6-8ed8-9474331cdc23 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:19.150523 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f5f03-7906-4bb6-8ed8-9474331cdc23 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:19.150793 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f5f03-7906-4bb6-8ed8-9474331cdc23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:19.161537 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f5f03-7906-4bb6-8ed8-9474331cdc23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:19.162957 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f5f03-7906-4bb6-8ed8-9474331cdc23 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:19.177514 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-951f5f03-7906-4bb6-8ed8-9474331cdc23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:53:19.177514 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:19.177514 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:53:19.177514 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:53:19.177514 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:53:19.177514 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:53:19.177514 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:53:19.177514 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:53:19.177514 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:53:19.177514 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:53:19.177514 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:53:19.177514 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:53:19.177514 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:53:19.177514 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:53:19.177514 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:53:19.178610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:53:19.178610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:53:19.178610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:53:19.178610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:53:19.178610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:53:19.178610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:53:19.178610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:53:19.178610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:53:19.178610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:53:19.178610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:53:19.178610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:53:19.178610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:53:19.178610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:53:19.178610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:53:19.178610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:53:19.178610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:53:19.178610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:53:19.179712 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:53:19.179712 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:53:19.179712 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:53:19.179712 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:53:19.179712 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 624/1265] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:53:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:53:19.184449 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f428905-5616-4709-b5e4-d42f7a480513 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:19.267323 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:19.287876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f428905-5616-4709-b5e4-d42f7a480513 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:19.324020 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b615fca-de6e-43c0-a5d5-718e521222f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:19.324647 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b615fca-de6e-43c0-a5d5-718e521222f7 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:19.324928 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b615fca-de6e-43c0-a5d5-718e521222f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:19.325197 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b615fca-de6e-43c0-a5d5-718e521222f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:19.325871 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b615fca-de6e-43c0-a5d5-718e521222f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:19.326171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b615fca-de6e-43c0-a5d5-718e521222f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:19.328043 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b615fca-de6e-43c0-a5d5-718e521222f7 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:19.339864 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 642/1266] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:53:19.347496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f5f03-7906-4bb6-8ed8-9474331cdc23 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:19.447662 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f5f03-7906-4bb6-8ed8-9474331cdc23 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:19.467197 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:19.481762 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2eb1bd-20c2-4e12-b220-681f2ff450ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:19.482390 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2eb1bd-20c2-4e12-b220-681f2ff450ba None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:19.482669 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2eb1bd-20c2-4e12-b220-681f2ff450ba None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:19.482939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2eb1bd-20c2-4e12-b220-681f2ff450ba None admin] PATH_INFO: `/v3/users/placement` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:19.491568 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2eb1bd-20c2-4e12-b220-681f2ff450ba None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:19.493042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2eb1bd-20c2-4e12-b220-681f2ff450ba None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:19.504951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-8c2eb1bd-20c2-4e12-b220-681f2ff450ba None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 19:53:19.504951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:19.504951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:53:19.504951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:53:19.504951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:53:19.504951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:53:19.504951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:53:19.504951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:53:19.504951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:53:19.504951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:53:19.504951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:53:19.504951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:53:19.504951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:53:19.504951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:53:19.504951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:53:19.504951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:53:19.506041 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:53:19.506041 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:53:19.506041 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:53:19.506041 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:53:19.506041 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:53:19.506041 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:53:19.506041 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:53:19.506041 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:53:19.506041 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:53:19.506041 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:53:19.506041 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:53:19.506041 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:53:19.506041 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:53:19.506041 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:53:19.506041 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:53:19.506041 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:53:19.506041 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:53:19.506041 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:53:19.507157 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:53:19.507157 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:53:19.507157 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:53:19.507157 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:53:19.507157 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:53:19.507157 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:53:19.507157 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:53:19.507157 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:53:19.507157 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 19:53:19.507157 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:53:19.507157 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 625/1267] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:53:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:53:19.512146 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b615fca-de6e-43c0-a5d5-718e521222f7 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:19.621277 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b615fca-de6e-43c0-a5d5-718e521222f7 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:19.660918 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d6952-81b4-46e1-8139-079e743b1242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:19.661571 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d6952-81b4-46e1-8139-079e743b1242 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:19.661865 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d6952-81b4-46e1-8139-079e743b1242 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:19.662138 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d6952-81b4-46e1-8139-079e743b1242 None admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:19.662969 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d6952-81b4-46e1-8139-079e743b1242 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:19.663240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d6952-81b4-46e1-8139-079e743b1242 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:19.664672 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d6952-81b4-46e1-8139-079e743b1242 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:19.673748 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:19.683128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 643/1268] 10.222.0.39 () {56 vars in 1143 bytes} [Fri Aug 5 19:53:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:19.689573 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2eb1bd-20c2-4e12-b220-681f2ff450ba None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:19.789405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2eb1bd-20c2-4e12-b220-681f2ff450ba None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:19.824354 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef3cbf-c45f-4b4a-885c-a24572d93c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:19.825120 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef3cbf-c45f-4b4a-885c-a24572d93c0b None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:19.825402 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef3cbf-c45f-4b4a-885c-a24572d93c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:19.825675 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef3cbf-c45f-4b4a-885c-a24572d93c0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:19.836436 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aef3cbf-c45f-4b4a-885c-a24572d93c0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:19.838040 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aef3cbf-c45f-4b4a-885c-a24572d93c0b None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:19.852595 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-9aef3cbf-c45f-4b4a-885c-a24572d93c0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:53:19.852595 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:19.852595 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:53:19.852595 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:53:19.852595 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:53:19.852595 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:53:19.852595 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:53:19.852595 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:53:19.852595 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:53:19.852595 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:53:19.852595 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:53:19.852595 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:53:19.852595 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:53:19.852595 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:53:19.852595 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:53:19.853677 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:53:19.853677 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:53:19.853677 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:53:19.853677 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:53:19.853677 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:53:19.853677 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:53:19.853677 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:53:19.853677 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:53:19.853677 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:53:19.853677 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:53:19.853677 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:53:19.853677 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:53:19.853677 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:53:19.853677 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:53:19.853677 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:53:19.853677 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:53:19.853677 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:53:19.854799 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:53:19.854799 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:53:19.854799 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:53:19.854799 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:53:19.854799 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 626/1269] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:53:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:53:19.860398 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d6952-81b4-46e1-8139-079e743b1242 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:19.959266 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d6952-81b4-46e1-8139-079e743b1242 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:19.993836 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195d478-aed2-48b2-b243-3415a548d6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:19.994440 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195d478-aed2-48b2-b243-3415a548d6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:19.994716 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195d478-aed2-48b2-b243-3415a548d6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:19.994987 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195d478-aed2-48b2-b243-3415a548d6a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:19.995662 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e195d478-aed2-48b2-b243-3415a548d6a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:19.995964 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e195d478-aed2-48b2-b243-3415a548d6a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:19.997341 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e195d478-aed2-48b2-b243-3415a548d6a7 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:20.008449 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 644/1270] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:20.014338 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef3cbf-c45f-4b4a-885c-a24572d93c0b None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:20.047559 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:20.116212 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef3cbf-c45f-4b4a-885c-a24572d93c0b None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:20.151000 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bfc24e-9a0f-489b-bedc-10a89315cf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:20.151633 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfc24e-9a0f-489b-bedc-10a89315cf0e None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:20.151913 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfc24e-9a0f-489b-bedc-10a89315cf0e None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:20.152184 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfc24e-9a0f-489b-bedc-10a89315cf0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:20.159696 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bfc24e-9a0f-489b-bedc-10a89315cf0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:20.161187 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bfc24e-9a0f-489b-bedc-10a89315cf0e None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:20.172851 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-59bfc24e-9a0f-489b-bedc-10a89315cf0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:53:20.172851 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:20.172851 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:53:20.172851 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:53:20.172851 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:53:20.172851 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:53:20.172851 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:53:20.172851 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:53:20.172851 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:53:20.172851 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:53:20.172851 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:53:20.172851 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:53:20.172851 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:53:20.172851 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:53:20.172851 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:53:20.173929 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:53:20.173929 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:53:20.173929 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:53:20.173929 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:53:20.173929 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:53:20.173929 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:53:20.173929 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:53:20.173929 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:53:20.173929 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:53:20.173929 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:53:20.173929 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:53:20.173929 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:53:20.173929 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:53:20.173929 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:53:20.173929 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:53:20.173929 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:53:20.173929 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:53:20.175058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:53:20.175058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:53:20.175058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:53:20.175058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:53:20.175058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:53:20.175058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:53:20.175058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:53:20.175058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:53:20.175058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 627/1271] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:53:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:53:20.179793 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195d478-aed2-48b2-b243-3415a548d6a7 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:20.241025 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:20.284084 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195d478-aed2-48b2-b243-3415a548d6a7 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:20.317711 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cac924-225b-4efb-98f6-65f1729af296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:20.318319 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cac924-225b-4efb-98f6-65f1729af296 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:20.318603 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cac924-225b-4efb-98f6-65f1729af296 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:20.318876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cac924-225b-4efb-98f6-65f1729af296 None admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:20.319613 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cac924-225b-4efb-98f6-65f1729af296 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:20.319919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cac924-225b-4efb-98f6-65f1729af296 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:20.321384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cac924-225b-4efb-98f6-65f1729af296 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:20.334039 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 645/1272] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:53:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:20.340240 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bfc24e-9a0f-489b-bedc-10a89315cf0e None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:20.432397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:20.443508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bfc24e-9a0f-489b-bedc-10a89315cf0e None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:20.477811 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc5b3e6-132e-462f-ad3f-726686e3c18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:20.478465 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5b3e6-132e-462f-ad3f-726686e3c18c None admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:20.478749 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5b3e6-132e-462f-ad3f-726686e3c18c None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:20.479025 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5b3e6-132e-462f-ad3f-726686e3c18c None admin] PATH_INFO: `/v3/projects/80379e2104644e98a2e85d4e4af1b365/users/c61e2c954f9a49abad9f3de17ee39f63/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:20.480959 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc5b3e6-132e-462f-ad3f-726686e3c18c None admin] RBAC: Authorizing `identity:create_grant(project_id=80379e2104644e98a2e85d4e4af1b365, user_id=c61e2c954f9a49abad9f3de17ee39f63, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:20.482376 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc5b3e6-132e-462f-ad3f-726686e3c18c None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:20.495671 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 628/1273] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Aug 5 19:53:20 2022] PUT /identity/v3/projects/80379e2104644e98a2e85d4e4af1b365/users/c61e2c954f9a49abad9f3de17ee39f63/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:53:22.500274 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a381dc-4a51-409e-85a3-59c38e70002f None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:22.500802 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a381dc-4a51-409e-85a3-59c38e70002f None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:22.500802 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a381dc-4a51-409e-85a3-59c38e70002f None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:22.502161 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 646/1274] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:53:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:53:22.507286 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b431b-389f-4ded-80da-66aaffdcee59 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:22.507725 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b431b-389f-4ded-80da-66aaffdcee59 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:22.507824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b431b-389f-4ded-80da-66aaffdcee59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:22.526370 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-e10b431b-389f-4ded-80da-66aaffdcee59 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:53:22.527560 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-e10b431b-389f-4ded-80da-66aaffdcee59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:53:22.595372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 629/1275] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:53:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:53:22.618833 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a381dc-4a51-409e-85a3-59c38e70002f None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:22.725383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a381dc-4a51-409e-85a3-59c38e70002f None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:22.742891 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:22.759735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cadf3-7ccf-4aa3-9b64-ffc08fe43b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:22.760294 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cadf3-7ccf-4aa3-9b64-ffc08fe43b10 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:22.760595 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cadf3-7ccf-4aa3-9b64-ffc08fe43b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:22.760874 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cadf3-7ccf-4aa3-9b64-ffc08fe43b10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:22.771400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1cadf3-7ccf-4aa3-9b64-ffc08fe43b10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:53:22.776300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-9f1cadf3-7ccf-4aa3-9b64-ffc08fe43b10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:53:22.776300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:22.776300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:53:22.776300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:53:22.776300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:53:22.776300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:53:22.776300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:53:22.776300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:53:22.776300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:53:22.776300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:53:22.776300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:53:22.776300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:53:22.776300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:53:22.776300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:53:22.776300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:53:22.776300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:53:22.777537 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:53:22.777537 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:53:22.777537 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:53:22.777537 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:53:22.777537 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:53:22.777537 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:53:22.777537 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:53:22.777537 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:53:22.777537 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:53:22.777537 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:53:22.777537 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:53:22.777537 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:53:22.777537 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:53:22.777537 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:53:22.777537 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:53:22.777537 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:53:22.778447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:53:22.778447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:53:22.778447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:53:22.778447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:53:22.778447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:53:22.778447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:53:22.778447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:53:22.778447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:53:22.778447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:53:22.778447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:53:22.778447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:53:22.778447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:53:22.778447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 647/1276] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Aug 5 19:53:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:53:22.784300 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b431b-389f-4ded-80da-66aaffdcee59 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:22.870870 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:22.890619 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b431b-389f-4ded-80da-66aaffdcee59 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:22.925481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9add1e-57f9-4280-b2e0-924eb14c3bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:22.926080 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9add1e-57f9-4280-b2e0-924eb14c3bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:22.926356 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9add1e-57f9-4280-b2e0-924eb14c3bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:22.926627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9add1e-57f9-4280-b2e0-924eb14c3bf0 None admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:22.927325 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9add1e-57f9-4280-b2e0-924eb14c3bf0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:22.927622 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9add1e-57f9-4280-b2e0-924eb14c3bf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:22.929004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9add1e-57f9-4280-b2e0-924eb14c3bf0 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:22.936870 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 630/1277] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Aug 5 19:53:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:53:22.942724 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cadf3-7ccf-4aa3-9b64-ffc08fe43b10 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:23.039733 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cadf3-7ccf-4aa3-9b64-ffc08fe43b10 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:23.072209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abda0e7-8f5e-4499-af91-10cce675585b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:23.072854 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abda0e7-8f5e-4499-af91-10cce675585b None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:23.073132 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abda0e7-8f5e-4499-af91-10cce675585b None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:23.073399 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abda0e7-8f5e-4499-af91-10cce675585b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:23.083791 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abda0e7-8f5e-4499-af91-10cce675585b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:23.085249 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abda0e7-8f5e-4499-af91-10cce675585b None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:23.099901 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-1abda0e7-8f5e-4499-af91-10cce675585b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:53:23.099901 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:23.099901 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:53:23.099901 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:53:23.099901 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:53:23.099901 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:53:23.099901 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:53:23.099901 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:53:23.099901 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:53:23.099901 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:53:23.099901 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:53:23.099901 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:53:23.099901 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:53:23.099901 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:53:23.099901 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:53:23.101009 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:53:23.101009 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:53:23.101009 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:53:23.101009 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:53:23.101009 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:53:23.101009 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:53:23.101009 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:53:23.101009 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:53:23.101009 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:53:23.101009 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:53:23.101009 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:53:23.101009 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:53:23.101009 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:53:23.101009 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:53:23.101009 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:53:23.101009 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:53:23.101009 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:53:23.102003 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:53:23.102003 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:53:23.102003 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:53:23.102003 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:53:23.102003 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 648/1278] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:53:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:53:23.106962 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9add1e-57f9-4280-b2e0-924eb14c3bf0 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:23.208714 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9add1e-57f9-4280-b2e0-924eb14c3bf0 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:23.241327 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e48e3-8714-4506-b1c2-af52fb9a84d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:23.241931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e48e3-8714-4506-b1c2-af52fb9a84d8 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:23.242212 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e48e3-8714-4506-b1c2-af52fb9a84d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:23.242486 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e48e3-8714-4506-b1c2-af52fb9a84d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:23.243172 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e48e3-8714-4506-b1c2-af52fb9a84d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:23.243473 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e48e3-8714-4506-b1c2-af52fb9a84d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:23.244852 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e48e3-8714-4506-b1c2-af52fb9a84d8 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:23.256867 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 631/1279] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:23.262649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abda0e7-8f5e-4499-af91-10cce675585b None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:23.294900 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:23.364426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abda0e7-8f5e-4499-af91-10cce675585b None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:23.398512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7dec0-8f99-4777-b396-463971c09621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:23.399140 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7dec0-8f99-4777-b396-463971c09621 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:23.399422 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7dec0-8f99-4777-b396-463971c09621 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:23.399699 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7dec0-8f99-4777-b396-463971c09621 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:23.407293 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c7dec0-8f99-4777-b396-463971c09621 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:23.408754 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c7dec0-8f99-4777-b396-463971c09621 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:23.419741 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-91c7dec0-8f99-4777-b396-463971c09621 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 19:53:23.419741 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:23.419741 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:53:23.419741 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:53:23.419741 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:53:23.419741 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:53:23.419741 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:53:23.419741 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:53:23.419741 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:53:23.419741 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:53:23.419741 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:53:23.419741 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:53:23.419741 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:53:23.419741 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:53:23.419741 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:53:23.419741 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:53:23.420915 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:53:23.420915 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:53:23.420915 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:53:23.420915 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:53:23.420915 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:53:23.420915 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:53:23.420915 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:53:23.420915 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:53:23.420915 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:53:23.420915 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:53:23.420915 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:53:23.420915 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:53:23.420915 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:53:23.420915 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:53:23.420915 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:53:23.420915 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:53:23.420915 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:53:23.420915 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:53:23.422101 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:53:23.422101 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:53:23.422101 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:53:23.422101 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:53:23.422101 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:53:23.422101 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:53:23.422101 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:53:23.422101 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:53:23.422101 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 19:53:23.422101 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:53:23.422101 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 649/1280] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:53:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:53:23.426927 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e48e3-8714-4506-b1c2-af52fb9a84d8 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:23.430699 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:23.525805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e48e3-8714-4506-b1c2-af52fb9a84d8 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:23.558602 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb946579-abd7-4114-8be4-31bc4e4f7355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:23.559135 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb946579-abd7-4114-8be4-31bc4e4f7355 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:23.559410 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb946579-abd7-4114-8be4-31bc4e4f7355 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:23.559680 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb946579-abd7-4114-8be4-31bc4e4f7355 None admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:23.560455 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb946579-abd7-4114-8be4-31bc4e4f7355 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:23.560782 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb946579-abd7-4114-8be4-31bc4e4f7355 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:23.563987 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb946579-abd7-4114-8be4-31bc4e4f7355 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:23.581155 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 632/1281] 10.222.0.39 () {56 vars in 1143 bytes} [Fri Aug 5 19:53:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:23.586974 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7dec0-8f99-4777-b396-463971c09621 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:23.683380 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:23.694167 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7dec0-8f99-4777-b396-463971c09621 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:23.727189 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892bf8bb-0143-4008-9915-bf54ff083dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:23.727795 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892bf8bb-0143-4008-9915-bf54ff083dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:23.728073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892bf8bb-0143-4008-9915-bf54ff083dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:23.728346 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892bf8bb-0143-4008-9915-bf54ff083dd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:23.795046 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892bf8bb-0143-4008-9915-bf54ff083dd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:23.797013 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892bf8bb-0143-4008-9915-bf54ff083dd9 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:23.884874 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:23.895780 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 650/1282] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Aug 5 19:53:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:53:23.901616 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb946579-abd7-4114-8be4-31bc4e4f7355 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:23.961944 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:24.007402 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb946579-abd7-4114-8be4-31bc4e4f7355 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:24.051300 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc73b71-4c03-48a1-aafb-67886c0b7131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:24.051928 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc73b71-4c03-48a1-aafb-67886c0b7131 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:24.052203 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc73b71-4c03-48a1-aafb-67886c0b7131 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:24.052472 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc73b71-4c03-48a1-aafb-67886c0b7131 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:24.062603 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc73b71-4c03-48a1-aafb-67886c0b7131 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:24.064023 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc73b71-4c03-48a1-aafb-67886c0b7131 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:24.078509 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-7bc73b71-4c03-48a1-aafb-67886c0b7131 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:53:24.078509 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:24.078509 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:53:24.078509 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:53:24.078509 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:53:24.078509 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:53:24.078509 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:53:24.078509 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:53:24.078509 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:53:24.078509 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:53:24.078509 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:53:24.078509 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:53:24.078509 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:53:24.078509 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:53:24.078509 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:53:24.079578 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:53:24.079578 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:53:24.079578 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:53:24.079578 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:53:24.079578 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:53:24.079578 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:53:24.079578 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:53:24.079578 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:53:24.079578 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:53:24.079578 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:53:24.079578 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:53:24.079578 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:53:24.079578 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:53:24.079578 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:53:24.079578 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:53:24.079578 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:53:24.079578 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:53:24.081080 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:53:24.081080 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:53:24.081080 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:53:24.081080 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:53:24.081080 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 633/1283] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:53:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:53:24.085463 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892bf8bb-0143-4008-9915-bf54ff083dd9 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:24.185571 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892bf8bb-0143-4008-9915-bf54ff083dd9 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:24.220302 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c9038-83a4-45c9-b9e3-45efe3d68640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:24.221076 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c9038-83a4-45c9-b9e3-45efe3d68640 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:24.221569 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c9038-83a4-45c9-b9e3-45efe3d68640 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:24.221843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c9038-83a4-45c9-b9e3-45efe3d68640 None admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:24.222521 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c9038-83a4-45c9-b9e3-45efe3d68640 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:24.222821 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c9038-83a4-45c9-b9e3-45efe3d68640 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:24.224173 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c9038-83a4-45c9-b9e3-45efe3d68640 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:24.235905 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 651/1284] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Aug 5 19:53:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:24.241713 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc73b71-4c03-48a1-aafb-67886c0b7131 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:24.343228 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc73b71-4c03-48a1-aafb-67886c0b7131 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:24.356513 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:24.377404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affef099-52a8-4407-b524-d83600922310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:24.378031 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affef099-52a8-4407-b524-d83600922310 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:24.378314 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affef099-52a8-4407-b524-d83600922310 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:24.378587 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affef099-52a8-4407-b524-d83600922310 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:24.385097 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affef099-52a8-4407-b524-d83600922310 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:24.386538 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affef099-52a8-4407-b524-d83600922310 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:24.399682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application [None req-affef099-52a8-4407-b524-d83600922310 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:53:24.399682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:24.399682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:53:24.399682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:53:24.399682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:53:24.399682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:53:24.399682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:53:24.399682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:53:24.399682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:53:24.399682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:53:24.399682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:53:24.399682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:53:24.399682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:53:24.399682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:53:24.399682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:53:24.400807 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:53:24.400807 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:53:24.400807 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:53:24.400807 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:53:24.400807 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:53:24.400807 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:53:24.400807 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:53:24.400807 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:53:24.400807 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application with Lock( Aug 05 19:53:24.400807 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:53:24.400807 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:53:24.400807 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:53:24.400807 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:53:24.400807 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:53:24.400807 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:53:24.400807 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:53:24.400807 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:53:24.402107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:53:24.402107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:53:24.402107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:53:24.402107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:53:24.402107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:53:24.402107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:53:24.402107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:53:24.402107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: ERROR keystone.server.flask.application Aug 05 19:53:24.402107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 634/1285] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Aug 5 19:53:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:53:24.407266 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c9038-83a4-45c9-b9e3-45efe3d68640 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:24.458088 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:24.511553 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c9038-83a4-45c9-b9e3-45efe3d68640 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:24.544370 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0e767-edb1-408d-a9fc-d3f87575bf19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:24.545002 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0e767-edb1-408d-a9fc-d3f87575bf19 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:24.545276 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0e767-edb1-408d-a9fc-d3f87575bf19 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:24.546114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0e767-edb1-408d-a9fc-d3f87575bf19 None admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:24.546847 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0e767-edb1-408d-a9fc-d3f87575bf19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:24.547149 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0e767-edb1-408d-a9fc-d3f87575bf19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:24.548598 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0e767-edb1-408d-a9fc-d3f87575bf19 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:24.560936 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 652/1286] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Aug 5 19:53:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:24.567581 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affef099-52a8-4407-b524-d83600922310 None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:24.673684 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affef099-52a8-4407-b524-d83600922310 None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:24.706472 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b8687-dc0c-47df-8641-f6db2934ca98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:24.707073 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b8687-dc0c-47df-8641-f6db2934ca98 None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:24.707353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b8687-dc0c-47df-8641-f6db2934ca98 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:24.707629 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b8687-dc0c-47df-8641-f6db2934ca98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:24.708407 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b8687-dc0c-47df-8641-f6db2934ca98 None admin] RBAC: Adding query filter params (role.id=89919f784ad144529b07c8ab92ee7d48, scope.project.id=80379e2104644e98a2e85d4e4af1b365, user.id=c61e2c954f9a49abad9f3de17ee39f63) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:24.708748 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b8687-dc0c-47df-8641-f6db2934ca98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:24.710175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b8687-dc0c-47df-8641-f6db2934ca98 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:24.715877 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 635/1287] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Aug 5 19:53:24 2022] GET /identity/v3/role_assignments?user.id=c61e2c954f9a49abad9f3de17ee39f63&scope.project.id=80379e2104644e98a2e85d4e4af1b365&role.id=89919f784ad144529b07c8ab92ee7d48 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:26.798291 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b41eef5-fc1c-4679-a613-63269468bf62 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:26.798821 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b41eef5-fc1c-4679-a613-63269468bf62 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:26.798821 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b41eef5-fc1c-4679-a613-63269468bf62 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:26.800220 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 653/1288] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:53:26.805969 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ebc4f-7b8d-4e10-b3d0-d61201701f8d None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:26.806380 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ebc4f-7b8d-4e10-b3d0-d61201701f8d None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:26.806524 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ebc4f-7b8d-4e10-b3d0-d61201701f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:26.825007 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-e05ebc4f-7b8d-4e10-b3d0-d61201701f8d None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:53:26.826173 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-e05ebc4f-7b8d-4e10-b3d0-d61201701f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:53:26.857162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:26.895347 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 636/1289] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:53:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:53:26.918047 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b41eef5-fc1c-4679-a613-63269468bf62 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:27.019690 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b41eef5-fc1c-4679-a613-63269468bf62 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:27.030100 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:27.053795 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b339d8a-7a89-4ad2-8dd4-54f46dd31adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:27.054354 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b339d8a-7a89-4ad2-8dd4-54f46dd31adf None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:27.054634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b339d8a-7a89-4ad2-8dd4-54f46dd31adf None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:27.054906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b339d8a-7a89-4ad2-8dd4-54f46dd31adf None admin] PATH_INFO: `/v3/services/placement` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:27.055528 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b339d8a-7a89-4ad2-8dd4-54f46dd31adf None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:27.056918 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b339d8a-7a89-4ad2-8dd4-54f46dd31adf None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:27.066752 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-7b339d8a-7a89-4ad2-8dd4-54f46dd31adf None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 19:53:27.066752 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:27.066752 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:53:27.066752 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:53:27.066752 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:53:27.066752 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:53:27.066752 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:53:27.066752 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:53:27.066752 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 19:53:27.066752 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:53:27.066752 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:53:27.066752 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:53:27.066752 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:27.066752 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:53:27.066752 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:53:27.066752 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:53:27.066752 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:53:27.068005 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:53:27.068005 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:53:27.068005 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:53:27.068005 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:53:27.068005 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:53:27.068005 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:53:27.068005 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:53:27.068005 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:53:27.068005 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:53:27.068005 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:53:27.068005 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:53:27.068005 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:53:27.068005 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:53:27.068005 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:53:27.068005 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:53:27.068005 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:53:27.068925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:53:27.068925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:53:27.068925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:53:27.068925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:53:27.068925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:53:27.068925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:53:27.068925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:53:27.068925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:53:27.068925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:53:27.068925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:53:27.068925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:53:27.068925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:53:27.068925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 19:53:27.068925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:53:27.068925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 654/1290] 10.222.0.39 () {56 vars in 1117 bytes} [Fri Aug 5 19:53:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:53:27.074013 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ebc4f-7b8d-4e10-b3d0-d61201701f8d None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:27.173384 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ebc4f-7b8d-4e10-b3d0-d61201701f8d None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:27.207752 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b0993c-a9bd-4a02-92f1-9bc60c2440ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:27.208356 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0993c-a9bd-4a02-92f1-9bc60c2440ff None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:27.208669 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0993c-a9bd-4a02-92f1-9bc60c2440ff None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:27.208947 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0993c-a9bd-4a02-92f1-9bc60c2440ff None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:27.209624 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b0993c-a9bd-4a02-92f1-9bc60c2440ff None admin] RBAC: Adding query filter params (name=placement) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:27.209930 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b0993c-a9bd-4a02-92f1-9bc60c2440ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:27.211286 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b0993c-a9bd-4a02-92f1-9bc60c2440ff None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:27.216074 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 637/1291] 10.222.0.39 () {56 vars in 1116 bytes} [Fri Aug 5 19:53:27 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:53:27.222178 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b339d8a-7a89-4ad2-8dd4-54f46dd31adf None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:27.319879 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b339d8a-7a89-4ad2-8dd4-54f46dd31adf None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:27.352704 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33448a1-49a1-4c58-a6c1-47a0cf91fb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:27.353332 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33448a1-49a1-4c58-a6c1-47a0cf91fb17 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:27.353610 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33448a1-49a1-4c58-a6c1-47a0cf91fb17 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:27.353881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33448a1-49a1-4c58-a6c1-47a0cf91fb17 None admin] PATH_INFO: `/v3/services` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:27.354542 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33448a1-49a1-4c58-a6c1-47a0cf91fb17 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:27.354840 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33448a1-49a1-4c58-a6c1-47a0cf91fb17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:27.356179 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33448a1-49a1-4c58-a6c1-47a0cf91fb17 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:27.360891 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 655/1292] 10.222.0.39 () {56 vars in 1116 bytes} [Fri Aug 5 19:53:27 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:29.387872 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027b534-2e19-405b-a65a-879bf09e9e3a None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:29.388393 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027b534-2e19-405b-a65a-879bf09e9e3a None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:29.388393 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027b534-2e19-405b-a65a-879bf09e9e3a None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:29.390198 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 638/1293] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:53:29.395626 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5edf40-ef2f-47a6-b8c6-d339875d49c6 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:29.396017 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5edf40-ef2f-47a6-b8c6-d339875d49c6 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:29.396107 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5edf40-ef2f-47a6-b8c6-d339875d49c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:29.414570 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-3f5edf40-ef2f-47a6-b8c6-d339875d49c6 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:53:29.415782 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-3f5edf40-ef2f-47a6-b8c6-d339875d49c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:53:29.445159 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:29.483994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 656/1294] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:53:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:53:29.507064 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1027b534-2e19-405b-a65a-879bf09e9e3a None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:29.538299 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:29.607699 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1027b534-2e19-405b-a65a-879bf09e9e3a None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:29.652462 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045295ff-127c-4cb6-bc20-d2ab019ff3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:29.653263 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045295ff-127c-4cb6-bc20-d2ab019ff3d0 None admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:29.653612 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045295ff-127c-4cb6-bc20-d2ab019ff3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:29.653928 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045295ff-127c-4cb6-bc20-d2ab019ff3d0 None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:29.655539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045295ff-127c-4cb6-bc20-d2ab019ff3d0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:29.657440 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045295ff-127c-4cb6-bc20-d2ab019ff3d0 None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:29.673435 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 639/1295] 10.222.0.39 () {60 vars in 1141 bytes} [Fri Aug 5 19:53:29 2022] POST /identity/v3/services => generated 246 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:53:31.676090 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028ad5d-c80f-46eb-98b6-94a0e87bd8c1 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:31.676644 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028ad5d-c80f-46eb-98b6-94a0e87bd8c1 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:31.676759 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028ad5d-c80f-46eb-98b6-94a0e87bd8c1 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:31.678031 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 657/1296] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:53:31.683242 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87bdf5-4102-4dc3-ac2b-2d7b5bb21bbb None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:31.683627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87bdf5-4102-4dc3-ac2b-2d7b5bb21bbb None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:31.683798 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87bdf5-4102-4dc3-ac2b-2d7b5bb21bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:31.702591 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-2b87bdf5-4102-4dc3-ac2b-2d7b5bb21bbb None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:53:31.703781 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-2b87bdf5-4102-4dc3-ac2b-2d7b5bb21bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:53:31.776240 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 640/1297] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:53:31 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:53:31.798946 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6028ad5d-c80f-46eb-98b6-94a0e87bd8c1 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:31.900842 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6028ad5d-c80f-46eb-98b6-94a0e87bd8c1 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:31.934767 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292655d-3dc3-49df-a073-147a04b0da7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:31.935356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292655d-3dc3-49df-a073-147a04b0da7a None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:31.936379 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292655d-3dc3-49df-a073-147a04b0da7a None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:31.936497 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292655d-3dc3-49df-a073-147a04b0da7a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:31.937114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292655d-3dc3-49df-a073-147a04b0da7a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:31.938476 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292655d-3dc3-49df-a073-147a04b0da7a None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:31.947400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-7292655d-3dc3-49df-a073-147a04b0da7a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 19:53:31.947400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:31.947400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:53:31.947400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:53:31.947400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:53:31.947400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:53:31.947400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:53:31.947400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:53:31.947400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 19:53:31.947400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:53:31.947400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:53:31.947400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:53:31.947400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:31.947400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:53:31.947400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:53:31.947400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:53:31.947400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:53:31.948671 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:53:31.948671 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:53:31.948671 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:53:31.948671 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:53:31.948671 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:53:31.948671 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:53:31.948671 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:53:31.948671 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:53:31.948671 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:53:31.948671 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:53:31.948671 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:53:31.948671 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:53:31.948671 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:53:31.948671 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:53:31.948671 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:53:31.948671 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:53:31.949794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:53:31.949794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:53:31.949794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:53:31.949794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:53:31.949794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:53:31.949794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:53:31.949794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:53:31.949794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:53:31.949794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:53:31.949794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:53:31.949794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:53:31.949794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:53:31.949794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 19:53:31.949794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:53:31.949794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 658/1298] 10.222.0.39 () {56 vars in 1117 bytes} [Fri Aug 5 19:53:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:53:31.954687 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87bdf5-4102-4dc3-ac2b-2d7b5bb21bbb None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:31.973394 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:32.054352 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87bdf5-4102-4dc3-ac2b-2d7b5bb21bbb None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:32.088715 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c90ffa-fa04-41e1-b939-75d5a2509e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:32.089337 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c90ffa-fa04-41e1-b939-75d5a2509e2b None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:32.089618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c90ffa-fa04-41e1-b939-75d5a2509e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:32.089894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c90ffa-fa04-41e1-b939-75d5a2509e2b None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:32.090568 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c90ffa-fa04-41e1-b939-75d5a2509e2b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:32.090873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c90ffa-fa04-41e1-b939-75d5a2509e2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:32.092517 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c90ffa-fa04-41e1-b939-75d5a2509e2b None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:32.097753 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 641/1299] 10.222.0.39 () {56 vars in 1116 bytes} [Fri Aug 5 19:53:31 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:32.103760 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292655d-3dc3-49df-a073-147a04b0da7a None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:32.136757 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:32.208120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292655d-3dc3-49df-a073-147a04b0da7a None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:32.241491 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68b207-9fc4-4336-9b78-f81afb2eafb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:32.242104 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68b207-9fc4-4336-9b78-f81afb2eafb0 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:32.242387 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68b207-9fc4-4336-9b78-f81afb2eafb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:32.242663 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68b207-9fc4-4336-9b78-f81afb2eafb0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:32.243415 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68b207-9fc4-4336-9b78-f81afb2eafb0 None admin] RBAC: Adding query filter params (interface=public, service_id=1676a4d05efe4dce8b87598775c0584c, region_id=RegionOne) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:32.243726 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68b207-9fc4-4336-9b78-f81afb2eafb0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:32.245125 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68b207-9fc4-4336-9b78-f81afb2eafb0 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:32.250553 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 659/1300] 10.222.0.39 () {56 vars in 1251 bytes} [Fri Aug 5 19:53:32 2022] GET /identity/v3/endpoints?service_id=1676a4d05efe4dce8b87598775c0584c&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:32.256186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c90ffa-fa04-41e1-b939-75d5a2509e2b None admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:32.338711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:32.359151 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c90ffa-fa04-41e1-b939-75d5a2509e2b None admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:32.392400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178aa6d-10c2-46bf-b126-7bc1a056ff7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:32.393025 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178aa6d-10c2-46bf-b126-7bc1a056ff7e None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:32.393301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178aa6d-10c2-46bf-b126-7bc1a056ff7e None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:32.393571 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178aa6d-10c2-46bf-b126-7bc1a056ff7e None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:32.394203 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3178aa6d-10c2-46bf-b126-7bc1a056ff7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:32.395544 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3178aa6d-10c2-46bf-b126-7bc1a056ff7e None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:32.402329 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 642/1301] 10.222.0.39 () {56 vars in 1087 bytes} [Fri Aug 5 19:53:32 2022] GET /identity/v3/services => generated 2191 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:53:34.401960 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacafd60-4f76-4cdf-983a-c97a57950e68 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:34.402467 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacafd60-4f76-4cdf-983a-c97a57950e68 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:34.402467 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacafd60-4f76-4cdf-983a-c97a57950e68 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:34.403784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 660/1302] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:53:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:53:34.409628 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa13727-5514-4475-9dd1-3b8e271f9e7d None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:34.410046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa13727-5514-4475-9dd1-3b8e271f9e7d None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:34.410178 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa13727-5514-4475-9dd1-3b8e271f9e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:34.428752 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-6aa13727-5514-4475-9dd1-3b8e271f9e7d None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:53:34.429934 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-6aa13727-5514-4475-9dd1-3b8e271f9e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:53:34.496154 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 643/1303] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:53:34 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:53:34.518903 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacafd60-4f76-4cdf-983a-c97a57950e68 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:34.617247 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:34.624928 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacafd60-4f76-4cdf-983a-c97a57950e68 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:34.668286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7a43e-bf4d-4352-9e57-e80797423ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:34.668833 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7a43e-bf4d-4352-9e57-e80797423ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:34.669157 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7a43e-bf4d-4352-9e57-e80797423ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:34.669494 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7a43e-bf4d-4352-9e57-e80797423ab2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:34.670747 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7a43e-bf4d-4352-9e57-e80797423ab2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:34.672227 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7a43e-bf4d-4352-9e57-e80797423ab2 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:34.682352 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application [None req-14d7a43e-bf4d-4352-9e57-e80797423ab2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 19:53:34.682352 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:34.682352 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:53:34.682352 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:53:34.682352 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:53:34.682352 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:53:34.682352 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:53:34.682352 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:53:34.682352 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 19:53:34.682352 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:53:34.682352 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:53:34.682352 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:53:34.682352 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:53:34.682352 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:53:34.682352 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:53:34.682352 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:53:34.682352 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:53:34.683460 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:53:34.683460 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:53:34.683460 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:53:34.683460 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:53:34.683460 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:53:34.683460 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:53:34.683460 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:53:34.683460 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:53:34.683460 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:53:34.683460 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:53:34.683460 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:53:34.683460 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:53:34.683460 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:53:34.683460 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:53:34.683460 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:53:34.683460 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:53:34.684325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:53:34.684325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application with Lock( Aug 05 19:53:34.684325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:53:34.684325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:53:34.684325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:53:34.684325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:53:34.684325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:53:34.684325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:53:34.684325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:53:34.684325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:53:34.684325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:53:34.684325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:53:34.684325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 19:53:34.684325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: ERROR keystone.server.flask.application Aug 05 19:53:34.684325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 661/1304] 10.222.0.39 () {56 vars in 1117 bytes} [Fri Aug 5 19:53:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:53:34.696439 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa13727-5514-4475-9dd1-3b8e271f9e7d None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:34.815140 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:34.838645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa13727-5514-4475-9dd1-3b8e271f9e7d None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:34.872076 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5918d566-7b72-484f-8af8-5efc2a73237e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:34.872808 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918d566-7b72-484f-8af8-5efc2a73237e None admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:34.873501 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918d566-7b72-484f-8af8-5efc2a73237e None admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:34.873501 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918d566-7b72-484f-8af8-5efc2a73237e None admin] PATH_INFO: `/v3/services` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:34.874572 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5918d566-7b72-484f-8af8-5efc2a73237e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:53:34.874879 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5918d566-7b72-484f-8af8-5efc2a73237e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:34.876229 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5918d566-7b72-484f-8af8-5efc2a73237e None admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:34.881459 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 644/1305] 10.222.0.39 () {56 vars in 1116 bytes} [Fri Aug 5 19:53:34 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:53:34.887330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7a43e-bf4d-4352-9e57-e80797423ab2 None admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:34.986844 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7a43e-bf4d-4352-9e57-e80797423ab2 None admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:35.020795 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146bc969-4857-4467-9fd9-7a936ad46e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:35.021398 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146bc969-4857-4467-9fd9-7a936ad46e34 None admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:35.021678 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146bc969-4857-4467-9fd9-7a936ad46e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:35.021948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146bc969-4857-4467-9fd9-7a936ad46e34 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:35.022698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146bc969-4857-4467-9fd9-7a936ad46e34 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:35.024085 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146bc969-4857-4467-9fd9-7a936ad46e34 None admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:35.043913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 662/1306] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 19:53:34 2022] POST /identity/v3/endpoints => generated 331 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:53:44.878405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=30 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:45.039715 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:54:39.126664 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6cd68-f8fb-4682-ab2b-87998af99626 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:39.127731 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6cd68-f8fb-4682-ab2b-87998af99626 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:39.128230 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6cd68-f8fb-4682-ab2b-87998af99626 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:39.135653 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da7cc5-6a72-46e8-9d2d-a24264ee7aa7 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:39.137287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da7cc5-6a72-46e8-9d2d-a24264ee7aa7 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:39.138097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da7cc5-6a72-46e8-9d2d-a24264ee7aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:39.140689 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 645/1307] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:54:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:54:39.233302 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-67da7cc5-6a72-46e8-9d2d-a24264ee7aa7 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:54:39.235112 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-67da7cc5-6a72-46e8-9d2d-a24264ee7aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:39.472601 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 663/1308] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:54:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:54:39.513765 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4349ae-61a7-4ead-a6f8-d935742bb58e None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:39.514361 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4349ae-61a7-4ead-a6f8-d935742bb58e None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:39.514854 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4349ae-61a7-4ead-a6f8-d935742bb58e None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:39.516720 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 646/1309] 10.222.0.39 () {54 vars in 965 bytes} [Fri Aug 5 19:54:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:54:39.524846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16542d25-b02b-4bcc-b7b5-db420aa6499e None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:39.525188 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16542d25-b02b-4bcc-b7b5-db420aa6499e None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:39.525467 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16542d25-b02b-4bcc-b7b5-db420aa6499e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:39.629493 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-16542d25-b02b-4bcc-b7b5-db420aa6499e None None] MFA Rules not processed for user `b1c75368c00e4b0cb7b3163728843479`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:54:39.630831 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-16542d25-b02b-4bcc-b7b5-db420aa6499e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:39.660305 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:54:39.735545 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 664/1310] 10.222.0.39 () {58 vars in 1062 bytes} [Fri Aug 5 19:54:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:54:39.744469 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4349ae-61a7-4ead-a6f8-d935742bb58e None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:54:39.746606 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-7b4349ae-61a7-4ead-a6f8-d935742bb58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:39.876703 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4349ae-61a7-4ead-a6f8-d935742bb58e None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:54:39.960305 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2b601-4fd5-497b-94ff-c5b4da9d9293 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:54:39.961373 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2b601-4fd5-497b-94ff-c5b4da9d9293 service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:39.961872 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2b601-4fd5-497b-94ff-c5b4da9d9293 service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:39.962332 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2b601-4fd5-497b-94ff-c5b4da9d9293 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:39.965462 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-90f2b601-4fd5-497b-94ff-c5b4da9d9293 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:40.042599 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:54:40.131125 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f2b601-4fd5-497b-94ff-c5b4da9d9293 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:54:40.132832 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f2b601-4fd5-497b-94ff-c5b4da9d9293 service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:54:40.163193 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 647/1311] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:54:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:54:42.642116 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df65db0-6268-4ddc-b586-1d930182c34c None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:42.642957 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df65db0-6268-4ddc-b586-1d930182c34c None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:42.642957 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df65db0-6268-4ddc-b586-1d930182c34c None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:42.644287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 665/1312] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:54:42.649660 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e86ee5-2e7d-4c11-90c5-d0fc45db1307 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:42.650298 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e86ee5-2e7d-4c11-90c5-d0fc45db1307 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:42.650298 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e86ee5-2e7d-4c11-90c5-d0fc45db1307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:42.749704 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-b6e86ee5-2e7d-4c11-90c5-d0fc45db1307 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:54:42.751273 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-b6e86ee5-2e7d-4c11-90c5-d0fc45db1307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:42.855258 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 648/1313] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:54:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:54:42.895748 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df65db0-6268-4ddc-b586-1d930182c34c None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:54:42.897268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-2df65db0-6268-4ddc-b586-1d930182c34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:43.006291 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df65db0-6268-4ddc-b586-1d930182c34c None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:54:43.081582 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:54:43.084779 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4660a-bb50-4348-958a-99457739ac62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:54:43.085516 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4660a-bb50-4348-958a-99457739ac62 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:43.086126 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4660a-bb50-4348-958a-99457739ac62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:43.086767 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4660a-bb50-4348-958a-99457739ac62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:43.088628 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-c2c4660a-bb50-4348-958a-99457739ac62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:43.210421 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4660a-bb50-4348-958a-99457739ac62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:54:43.211785 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4660a-bb50-4348-958a-99457739ac62 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:54:43.229916 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 666/1314] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:54:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:54:45.582639 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625c4c5-18ae-41d6-98b9-e4a3b2c1f33e None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:45.583214 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625c4c5-18ae-41d6-98b9-e4a3b2c1f33e None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:45.583214 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625c4c5-18ae-41d6-98b9-e4a3b2c1f33e None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:45.584768 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 649/1315] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:54:45.589957 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3b204-aed1-4639-947c-b3337c68ed41 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:45.590506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3b204-aed1-4639-947c-b3337c68ed41 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:45.590506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3b204-aed1-4639-947c-b3337c68ed41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:45.661254 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-dbd3b204-aed1-4639-947c-b3337c68ed41 None None] MFA Rules not processed for user `9a89bd577d164d69948fbb5e8b571169`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:54:45.662462 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-dbd3b204-aed1-4639-947c-b3337c68ed41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:45.715442 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:54:45.760629 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 667/1316] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:54:45 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:54:45.814048 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda834f-f6c1-4cc1-a96d-450b83212314 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:45.814337 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda834f-f6c1-4cc1-a96d-450b83212314 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:45.814621 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda834f-f6c1-4cc1-a96d-450b83212314 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:45.816005 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 650/1317] 10.222.0.39 () {54 vars in 965 bytes} [Fri Aug 5 19:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:54:45.821809 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a3b69-1a3f-48bd-829d-025cb906fe48 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:45.822094 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a3b69-1a3f-48bd-829d-025cb906fe48 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:45.822373 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a3b69-1a3f-48bd-829d-025cb906fe48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:45.899864 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-a42a3b69-1a3f-48bd-829d-025cb906fe48 None None] MFA Rules not processed for user `b1c75368c00e4b0cb7b3163728843479`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:54:45.901119 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-a42a3b69-1a3f-48bd-829d-025cb906fe48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:45.977566 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 668/1318] 10.222.0.39 () {58 vars in 1062 bytes} [Fri Aug 5 19:54:45 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:54:45.986955 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda834f-f6c1-4cc1-a96d-450b83212314 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:54:45.988536 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-ebda834f-f6c1-4cc1-a96d-450b83212314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:45.999621 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:54:46.097760 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda834f-f6c1-4cc1-a96d-450b83212314 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:54:46.171524 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476a18f-9ad4-4984-b505-f5b2fc83c9b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:54:46.172068 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476a18f-9ad4-4984-b505-f5b2fc83c9b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:46.172345 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476a18f-9ad4-4984-b505-f5b2fc83c9b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:46.172644 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476a18f-9ad4-4984-b505-f5b2fc83c9b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:46.174271 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-c476a18f-9ad4-4984-b505-f5b2fc83c9b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:46.236782 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:54:46.284886 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476a18f-9ad4-4984-b505-f5b2fc83c9b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:54:46.286328 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476a18f-9ad4-4984-b505-f5b2fc83c9b6 service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:54:46.304239 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 651/1319] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:54:45 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:54:49.323298 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36652c1c-f263-4f7d-bcfe-e5c85fa3fd07 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:49.323822 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36652c1c-f263-4f7d-bcfe-e5c85fa3fd07 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:49.323822 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36652c1c-f263-4f7d-bcfe-e5c85fa3fd07 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:49.325546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 669/1320] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:54:49.330953 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a4081-21f8-43c2-bb8a-dfc2f891bedb None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:49.331366 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a4081-21f8-43c2-bb8a-dfc2f891bedb None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:49.331586 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a4081-21f8-43c2-bb8a-dfc2f891bedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:49.399191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-f50a4081-21f8-43c2-bb8a-dfc2f891bedb None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:54:49.400412 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-f50a4081-21f8-43c2-bb8a-dfc2f891bedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:49.498306 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 652/1321] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:54:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:54:49.539248 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36652c1c-f263-4f7d-bcfe-e5c85fa3fd07 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:54:49.540824 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-36652c1c-f263-4f7d-bcfe-e5c85fa3fd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:49.594566 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:54:49.659073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36652c1c-f263-4f7d-bcfe-e5c85fa3fd07 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:54:49.734873 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ed4d1-2e38-4401-95ce-6ddc0b88c32a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:54:49.735476 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ed4d1-2e38-4401-95ce-6ddc0b88c32a service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:49.735750 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ed4d1-2e38-4401-95ce-6ddc0b88c32a service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:49.736019 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ed4d1-2e38-4401-95ce-6ddc0b88c32a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:49.737658 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-385ed4d1-2e38-4401-95ce-6ddc0b88c32a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:49.834467 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:54:49.848869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ed4d1-2e38-4401-95ce-6ddc0b88c32a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:54:49.850354 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ed4d1-2e38-4401-95ce-6ddc0b88c32a service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:54:49.868314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 670/1322] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:54:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:54:52.670384 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c91ef-cded-432b-a3ab-c0db76a6ba6d None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:52.670958 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c91ef-cded-432b-a3ab-c0db76a6ba6d None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:52.670958 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c91ef-cded-432b-a3ab-c0db76a6ba6d None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:52.672360 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 653/1323] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:54:52.677986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c97e00-2ef4-4fef-bce1-e2e485aa0b38 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:52.678625 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c97e00-2ef4-4fef-bce1-e2e485aa0b38 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:52.678625 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c97e00-2ef4-4fef-bce1-e2e485aa0b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:52.749949 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-48c97e00-2ef4-4fef-bce1-e2e485aa0b38 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:54:52.751164 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-48c97e00-2ef4-4fef-bce1-e2e485aa0b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:52.865231 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 671/1324] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:54:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:54:52.904605 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c91ef-cded-432b-a3ab-c0db76a6ba6d None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:54:52.906176 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-093c91ef-cded-432b-a3ab-c0db76a6ba6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:52.927489 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:54:53.019849 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c91ef-cded-432b-a3ab-c0db76a6ba6d None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:54:53.094998 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58423a78-423e-436c-bb0f-68819ec0e122 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:54:53.095614 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58423a78-423e-436c-bb0f-68819ec0e122 service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:53.095894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58423a78-423e-436c-bb0f-68819ec0e122 service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:53.096169 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58423a78-423e-436c-bb0f-68819ec0e122 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:53.097817 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-58423a78-423e-436c-bb0f-68819ec0e122 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:53.175246 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:54:53.216571 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58423a78-423e-436c-bb0f-68819ec0e122 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:54:53.218019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58423a78-423e-436c-bb0f-68819ec0e122 service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:54:53.238446 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 654/1325] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:54:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:54:55.324829 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ded21-557c-4225-bbba-ed5e7953cc27 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:55.324829 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ded21-557c-4225-bbba-ed5e7953cc27 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:55.324829 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ded21-557c-4225-bbba-ed5e7953cc27 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:55.326099 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 672/1326] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:54:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:54:55.331464 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0307c-8ce0-4beb-91f8-fc6ead455641 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:55.331903 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0307c-8ce0-4beb-91f8-fc6ead455641 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:55.332136 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0307c-8ce0-4beb-91f8-fc6ead455641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:55.398561 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-29a0307c-8ce0-4beb-91f8-fc6ead455641 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:54:55.399778 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-29a0307c-8ce0-4beb-91f8-fc6ead455641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:55.499382 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 655/1327] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:54:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:54:55.536658 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6ded21-557c-4225-bbba-ed5e7953cc27 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:54:55.538242 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-4a6ded21-557c-4225-bbba-ed5e7953cc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:55.601662 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:54:55.650483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6ded21-557c-4225-bbba-ed5e7953cc27 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:54:55.728767 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad5f024-5201-4898-b943-a035d3c5f740 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:54:55.729777 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5f024-5201-4898-b943-a035d3c5f740 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:55.729955 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5f024-5201-4898-b943-a035d3c5f740 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:55.730219 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5f024-5201-4898-b943-a035d3c5f740 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:55.733643 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-cad5f024-5201-4898-b943-a035d3c5f740 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:55.877840 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:54:55.880631 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad5f024-5201-4898-b943-a035d3c5f740 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:54:55.882765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad5f024-5201-4898-b943-a035d3c5f740 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:54:55.904019 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 673/1328] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:54:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:54:58.014920 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69acbc1-0c56-408a-a5db-d42f2bfb5962 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:58.015452 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69acbc1-0c56-408a-a5db-d42f2bfb5962 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:58.015541 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69acbc1-0c56-408a-a5db-d42f2bfb5962 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:58.016918 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 656/1329] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:54:58.022521 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89596b3-76f8-4e56-abcf-4fc115b59e9e None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:58.022961 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89596b3-76f8-4e56-abcf-4fc115b59e9e None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:58.023078 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89596b3-76f8-4e56-abcf-4fc115b59e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:58.094984 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-f89596b3-76f8-4e56-abcf-4fc115b59e9e None None] MFA Rules not processed for user `9a89bd577d164d69948fbb5e8b571169`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:54:58.096256 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-f89596b3-76f8-4e56-abcf-4fc115b59e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:58.204479 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 674/1330] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:54:58 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:54:58.244327 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69acbc1-0c56-408a-a5db-d42f2bfb5962 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:54:58.246052 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-e69acbc1-0c56-408a-a5db-d42f2bfb5962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:58.278258 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:54:58.352037 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69acbc1-0c56-408a-a5db-d42f2bfb5962 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:54:58.428147 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc366bc7-ba49-49bd-a329-34695faf04ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:54:58.428849 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc366bc7-ba49-49bd-a329-34695faf04ac service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:58.429253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc366bc7-ba49-49bd-a329-34695faf04ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:58.429354 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc366bc7-ba49-49bd-a329-34695faf04ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:58.430951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-bc366bc7-ba49-49bd-a329-34695faf04ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:58.509554 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:54:58.542477 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc366bc7-ba49-49bd-a329-34695faf04ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:54:58.544401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc366bc7-ba49-49bd-a329-34695faf04ac service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:54:58.581166 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 657/1331] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:54:58 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:55:00.919995 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ef5fd-7dbd-489f-ae45-b79a79e4d354 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:00.920564 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ef5fd-7dbd-489f-ae45-b79a79e4d354 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:00.920564 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ef5fd-7dbd-489f-ae45-b79a79e4d354 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:00.921923 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 675/1332] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:55:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:55:00.927053 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e068e5-4881-4da8-a336-08b3cc75f211 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:00.927493 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e068e5-4881-4da8-a336-08b3cc75f211 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:00.927623 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e068e5-4881-4da8-a336-08b3cc75f211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:00.994400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-f6e068e5-4881-4da8-a336-08b3cc75f211 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:00.995613 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-f6e068e5-4881-4da8-a336-08b3cc75f211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:01.096213 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 658/1333] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:55:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:55:01.133783 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ef5fd-7dbd-489f-ae45-b79a79e4d354 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:01.135433 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-848ef5fd-7dbd-489f-ae45-b79a79e4d354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:01.213563 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:01.250972 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ef5fd-7dbd-489f-ae45-b79a79e4d354 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:01.336352 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f21f76a-ba2d-4706-8013-932bd36aa2ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:01.336993 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21f76a-ba2d-4706-8013-932bd36aa2ad service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:01.337270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21f76a-ba2d-4706-8013-932bd36aa2ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:01.337541 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21f76a-ba2d-4706-8013-932bd36aa2ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:01.339164 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-7f21f76a-ba2d-4706-8013-932bd36aa2ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:01.454395 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f21f76a-ba2d-4706-8013-932bd36aa2ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:01.455752 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f21f76a-ba2d-4706-8013-932bd36aa2ad service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:01.466519 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:01.478192 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 676/1334] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:55:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:55:03.611554 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8fdf2-f19a-4ad9-8840-cbee720129b4 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:03.612205 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8fdf2-f19a-4ad9-8840-cbee720129b4 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:03.612205 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8fdf2-f19a-4ad9-8840-cbee720129b4 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:03.613484 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 659/1335] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:55:03.618829 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d39e0-2596-439e-abb3-837bbca45e8e None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:03.619266 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d39e0-2596-439e-abb3-837bbca45e8e None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:03.619390 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d39e0-2596-439e-abb3-837bbca45e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:03.687287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-053d39e0-2596-439e-abb3-837bbca45e8e None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:03.688492 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-053d39e0-2596-439e-abb3-837bbca45e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:03.784977 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 677/1336] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:55:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:55:03.827086 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8fdf2-f19a-4ad9-8840-cbee720129b4 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:03.828645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-a6d8fdf2-f19a-4ad9-8840-cbee720129b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:03.869936 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:03.934833 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8fdf2-f19a-4ad9-8840-cbee720129b4 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:04.007533 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5735f513-c7dc-479a-8fcb-088b0043383f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:04.008143 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735f513-c7dc-479a-8fcb-088b0043383f service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:04.008424 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735f513-c7dc-479a-8fcb-088b0043383f service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:04.008728 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735f513-c7dc-479a-8fcb-088b0043383f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:04.010345 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-5735f513-c7dc-479a-8fcb-088b0043383f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:04.093801 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:04.120855 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5735f513-c7dc-479a-8fcb-088b0043383f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:04.123637 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5735f513-c7dc-479a-8fcb-088b0043383f service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:04.142614 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 660/1337] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:55:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:55:06.589832 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28871f-7645-4f3c-953b-65af1c2d1b26 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:06.590686 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28871f-7645-4f3c-953b-65af1c2d1b26 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:06.590686 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28871f-7645-4f3c-953b-65af1c2d1b26 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:06.591927 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 678/1338] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:55:06.597285 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29357cc-1dc7-4736-a6c4-4e63dcfa5ec2 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:06.597705 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29357cc-1dc7-4736-a6c4-4e63dcfa5ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:06.597847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29357cc-1dc7-4736-a6c4-4e63dcfa5ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:06.665604 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-d29357cc-1dc7-4736-a6c4-4e63dcfa5ec2 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:06.666827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-d29357cc-1dc7-4736-a6c4-4e63dcfa5ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:06.765387 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 661/1339] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:55:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:55:06.802824 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e28871f-7645-4f3c-953b-65af1c2d1b26 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:06.804459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-8e28871f-7645-4f3c-953b-65af1c2d1b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:06.883097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:06.912527 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e28871f-7645-4f3c-953b-65af1c2d1b26 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:06.986194 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d913e1-0101-4e4c-a214-82ce664c907b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:06.986805 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d913e1-0101-4e4c-a214-82ce664c907b service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:06.987084 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d913e1-0101-4e4c-a214-82ce664c907b service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:06.987354 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d913e1-0101-4e4c-a214-82ce664c907b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:06.989013 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-29d913e1-0101-4e4c-a214-82ce664c907b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:07.101248 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d913e1-0101-4e4c-a214-82ce664c907b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:07.102701 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d913e1-0101-4e4c-a214-82ce664c907b service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:07.121018 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 679/1340] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:55:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:55:11.207064 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfabff-c5aa-4023-a675-018ceb8e4bea None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:11.207991 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfabff-c5aa-4023-a675-018ceb8e4bea None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:11.207991 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfabff-c5aa-4023-a675-018ceb8e4bea None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:11.209759 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 662/1341] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:55:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:55:11.215677 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44aec1-b9e6-4b5c-963d-47c5d3399f77 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:11.216181 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44aec1-b9e6-4b5c-963d-47c5d3399f77 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:11.216340 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44aec1-b9e6-4b5c-963d-47c5d3399f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:11.221928 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:11.283705 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-0f44aec1-b9e6-4b5c-963d-47c5d3399f77 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:11.285050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-0f44aec1-b9e6-4b5c-963d-47c5d3399f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:11.387429 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 680/1342] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:55:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:55:11.426425 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddfabff-c5aa-4023-a675-018ceb8e4bea None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:11.427972 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-eddfabff-c5aa-4023-a675-018ceb8e4bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:11.479103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:11.546010 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddfabff-c5aa-4023-a675-018ceb8e4bea None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:11.623251 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb305a6-adaa-4220-a711-24e03153dfdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:11.624023 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb305a6-adaa-4220-a711-24e03153dfdd service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:11.624380 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb305a6-adaa-4220-a711-24e03153dfdd service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:11.624789 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb305a6-adaa-4220-a711-24e03153dfdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:11.626624 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-ddb305a6-adaa-4220-a711-24e03153dfdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:11.731256 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:11.748760 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb305a6-adaa-4220-a711-24e03153dfdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:11.750195 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb305a6-adaa-4220-a711-24e03153dfdd service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:11.770191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 663/1343] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:55:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:55:14.487438 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992684d-5abb-4d76-bf5e-913910de153e None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:14.487898 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992684d-5abb-4d76-bf5e-913910de153e None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:14.488010 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992684d-5abb-4d76-bf5e-913910de153e None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:14.489361 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 681/1344] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:55:14.495036 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c3bf83-6c7f-433f-bf5c-794d648a064b None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:14.495484 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c3bf83-6c7f-433f-bf5c-794d648a064b None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:14.495581 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c3bf83-6c7f-433f-bf5c-794d648a064b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:14.564981 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-59c3bf83-6c7f-433f-bf5c-794d648a064b None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:14.566179 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-59c3bf83-6c7f-433f-bf5c-794d648a064b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:14.674715 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 664/1345] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:55:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:55:14.712129 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0992684d-5abb-4d76-bf5e-913910de153e None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:14.713681 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-0992684d-5abb-4d76-bf5e-913910de153e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:14.802523 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:14.820823 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0992684d-5abb-4d76-bf5e-913910de153e None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:14.896607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79af0b-52e8-43cb-8a1f-a62df8b57115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:14.897249 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79af0b-52e8-43cb-8a1f-a62df8b57115 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:14.897532 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79af0b-52e8-43cb-8a1f-a62df8b57115 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:14.897807 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79af0b-52e8-43cb-8a1f-a62df8b57115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:14.899434 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-0d79af0b-52e8-43cb-8a1f-a62df8b57115 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:15.023469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79af0b-52e8-43cb-8a1f-a62df8b57115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:15.024952 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79af0b-52e8-43cb-8a1f-a62df8b57115 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:15.043606 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 682/1346] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:55:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:55:19.301175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae1d98-31f1-4a8d-8235-1c6dcf49a4b2 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:19.301723 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae1d98-31f1-4a8d-8235-1c6dcf49a4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:19.301723 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae1d98-31f1-4a8d-8235-1c6dcf49a4b2 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:19.303377 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 665/1347] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:55:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:55:19.309516 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584be6b-0599-4c21-a98e-deeb6a38da17 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:19.309976 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584be6b-0599-4c21-a98e-deeb6a38da17 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:19.309976 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584be6b-0599-4c21-a98e-deeb6a38da17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:19.329207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:19.397921 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-b584be6b-0599-4c21-a98e-deeb6a38da17 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:19.399222 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-b584be6b-0599-4c21-a98e-deeb6a38da17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:19.512394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 683/1348] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:55:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:55:19.552390 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae1d98-31f1-4a8d-8235-1c6dcf49a4b2 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:19.553983 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-73ae1d98-31f1-4a8d-8235-1c6dcf49a4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:19.628391 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:19.680480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae1d98-31f1-4a8d-8235-1c6dcf49a4b2 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:19.760080 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b192ab7-27c0-4193-ae9d-80bc160c76ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:19.760746 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b192ab7-27c0-4193-ae9d-80bc160c76ef service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:19.761033 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b192ab7-27c0-4193-ae9d-80bc160c76ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:19.761311 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b192ab7-27c0-4193-ae9d-80bc160c76ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:19.763253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-0b192ab7-27c0-4193-ae9d-80bc160c76ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:19.880329 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:19.886241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b192ab7-27c0-4193-ae9d-80bc160c76ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:19.888859 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b192ab7-27c0-4193-ae9d-80bc160c76ef service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:19.909307 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 666/1349] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:55:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:55:24.848750 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8595c3-5bb2-4cc2-81f2-28fbb7b38d4c None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:24.849171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8595c3-5bb2-4cc2-81f2-28fbb7b38d4c None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:24.849333 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8595c3-5bb2-4cc2-81f2-28fbb7b38d4c None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:24.851095 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 684/1350] 10.222.0.39 () {54 vars in 921 bytes} [Fri Aug 5 19:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:55:24.856944 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811f608-2832-42a8-bea1-30fc57de12a7 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:24.857789 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811f608-2832-42a8-bea1-30fc57de12a7 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:24.858282 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811f608-2832-42a8-bea1-30fc57de12a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:25.006714 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-7811f608-2832-42a8-bea1-30fc57de12a7 None None] MFA Rules not processed for user `c61e2c954f9a49abad9f3de17ee39f63`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:25.007959 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-7811f608-2832-42a8-bea1-30fc57de12a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:25.141152 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 667/1351] 10.222.0.39 () {58 vars in 1018 bytes} [Fri Aug 5 19:55:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:55:29.425553 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dda05c-e3c7-4a05-a884-f2089aed57ca None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:29.426143 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dda05c-e3c7-4a05-a884-f2089aed57ca None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:29.426143 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dda05c-e3c7-4a05-a884-f2089aed57ca None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:29.427628 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 685/1352] 10.222.0.39 () {54 vars in 921 bytes} [Fri Aug 5 19:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:55:29.432641 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e35175-9357-4ea4-bcb6-95eaff7a4b3b None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:29.433467 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e35175-9357-4ea4-bcb6-95eaff7a4b3b None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:29.434557 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e35175-9357-4ea4-bcb6-95eaff7a4b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:29.498894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:29.510304 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=59 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:29.531390 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-29e35175-9357-4ea4-bcb6-95eaff7a4b3b None None] MFA Rules not processed for user `c61e2c954f9a49abad9f3de17ee39f63`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:29.533053 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-29e35175-9357-4ea4-bcb6-95eaff7a4b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:29.663200 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 668/1353] 10.222.0.39 () {58 vars in 1018 bytes} [Fri Aug 5 19:55:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:55:31.024003 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929bced-7690-4e8b-9138-5d61f7202152 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:31.024527 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929bced-7690-4e8b-9138-5d61f7202152 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:31.024527 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929bced-7690-4e8b-9138-5d61f7202152 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:31.026117 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 686/1354] 10.222.0.39 () {54 vars in 921 bytes} [Fri Aug 5 19:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:55:31.031075 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e4e77-b981-4f07-9e05-6213eb0a7e43 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:31.031555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e4e77-b981-4f07-9e05-6213eb0a7e43 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:31.031653 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e4e77-b981-4f07-9e05-6213eb0a7e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:31.115469 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-cf7e4e77-b981-4f07-9e05-6213eb0a7e43 None None] MFA Rules not processed for user `c61e2c954f9a49abad9f3de17ee39f63`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:31.116731 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-cf7e4e77-b981-4f07-9e05-6213eb0a7e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:31.165505 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:31.251645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 669/1355] 10.222.0.39 () {58 vars in 1018 bytes} [Fri Aug 5 19:55:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:55:34.872487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60ef24-4389-4c5f-a2ee-d625bcc4df35 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:34.873092 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60ef24-4389-4c5f-a2ee-d625bcc4df35 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:34.873092 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60ef24-4389-4c5f-a2ee-d625bcc4df35 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:34.874896 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 687/1356] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:55:34.880494 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6ba67-8269-4585-9597-31e1f1ea3a30 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:34.880979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6ba67-8269-4585-9597-31e1f1ea3a30 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:34.881088 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6ba67-8269-4585-9597-31e1f1ea3a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:34.968200 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-c8d6ba67-8269-4585-9597-31e1f1ea3a30 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:34.970870 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-c8d6ba67-8269-4585-9597-31e1f1ea3a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:35.092659 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 670/1357] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:55:35.150629 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371692e-d3e9-4bbb-8048-6ad38e9b0713 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:35.151312 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371692e-d3e9-4bbb-8048-6ad38e9b0713 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:35.151811 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371692e-d3e9-4bbb-8048-6ad38e9b0713 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:35.155257 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 688/1358] 10.222.0.39 () {54 vars in 962 bytes} [Fri Aug 5 19:55:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:55:35.163588 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09ae76-5021-4aab-9af2-ffac9212fc56 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:35.163879 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09ae76-5021-4aab-9af2-ffac9212fc56 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:35.164162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09ae76-5021-4aab-9af2-ffac9212fc56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:35.186002 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:35.271167 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-8e09ae76-5021-4aab-9af2-ffac9212fc56 None None] MFA Rules not processed for user `95e9f085e4f04d118373b8abff27543a`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:35.272362 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-8e09ae76-5021-4aab-9af2-ffac9212fc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:35.404099 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 671/1359] 10.222.0.39 () {58 vars in 1059 bytes} [Fri Aug 5 19:55:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:55:35.411682 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4371692e-d3e9-4bbb-8048-6ad38e9b0713 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:35.413240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-4371692e-d3e9-4bbb-8048-6ad38e9b0713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:35.567309 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4371692e-d3e9-4bbb-8048-6ad38e9b0713 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:35.647801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:35.691291 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c148c-4572-4147-8533-a0fabde1b879 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:35.691910 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c148c-4572-4147-8533-a0fabde1b879 service nova] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:35.692188 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c148c-4572-4147-8533-a0fabde1b879 service nova] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:35.692459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c148c-4572-4147-8533-a0fabde1b879 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:35.694095 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-a23c148c-4572-4147-8533-a0fabde1b879 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:35.792405 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c148c-4572-4147-8533-a0fabde1b879 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:35.794208 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c148c-4572-4147-8533-a0fabde1b879 service nova] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:35.817317 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 689/1360] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:55:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:55:38.098301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303e191-4575-479d-89c8-2add68a8c832 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:38.098838 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303e191-4575-479d-89c8-2add68a8c832 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:38.099194 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303e191-4575-479d-89c8-2add68a8c832 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:38.101047 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 672/1361] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:55:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:55:38.106850 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d904bb8-ad56-47d4-8d92-e0065d428ce0 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:38.107289 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d904bb8-ad56-47d4-8d92-e0065d428ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:38.107380 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d904bb8-ad56-47d4-8d92-e0065d428ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:38.189883 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-5d904bb8-ad56-47d4-8d92-e0065d428ce0 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:38.191194 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5d904bb8-ad56-47d4-8d92-e0065d428ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:38.233936 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:38.304664 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 690/1362] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:55:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:55:38.353671 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303e191-4575-479d-89c8-2add68a8c832 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:38.355239 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-b303e191-4575-479d-89c8-2add68a8c832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:38.432641 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:38.537914 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303e191-4575-479d-89c8-2add68a8c832 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:38.713544 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18afd945-71cd-49ec-9d4a-9e903eabd371 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:38.714275 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18afd945-71cd-49ec-9d4a-9e903eabd371 service nova] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:38.714657 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18afd945-71cd-49ec-9d4a-9e903eabd371 service nova] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:38.715700 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18afd945-71cd-49ec-9d4a-9e903eabd371 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:38.717520 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-18afd945-71cd-49ec-9d4a-9e903eabd371 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:38.741780 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:38.823137 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18afd945-71cd-49ec-9d4a-9e903eabd371 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:38.825347 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18afd945-71cd-49ec-9d4a-9e903eabd371 service nova] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:38.844850 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 673/1363] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:55:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:55:41.256505 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577c1bd-d08f-4c35-adf3-ee3197c647c6 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:41.257097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577c1bd-d08f-4c35-adf3-ee3197c647c6 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:41.257436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577c1bd-d08f-4c35-adf3-ee3197c647c6 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:41.258985 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 691/1364] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:55:41.265636 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93f5ad-b5e0-410a-9c21-e3850a266362 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:41.266151 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93f5ad-b5e0-410a-9c21-e3850a266362 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:41.266248 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93f5ad-b5e0-410a-9c21-e3850a266362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:41.369439 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-0f93f5ad-b5e0-410a-9c21-e3850a266362 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:41.371020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-0f93f5ad-b5e0-410a-9c21-e3850a266362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:41.498190 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:41.512911 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 674/1365] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:55:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:55:41.564956 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd75d2-8410-4aa9-bbf5-bf43ad9c71cd None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:41.565553 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd75d2-8410-4aa9-bbf5-bf43ad9c71cd None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:41.565944 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd75d2-8410-4aa9-bbf5-bf43ad9c71cd None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:41.568403 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 692/1366] 10.222.0.39 () {54 vars in 962 bytes} [Fri Aug 5 19:55:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:55:41.575785 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedda00d-8845-4ea6-a6be-d97f67891d9b None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:41.576158 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedda00d-8845-4ea6-a6be-d97f67891d9b None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:41.576502 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedda00d-8845-4ea6-a6be-d97f67891d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:41.664055 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-eedda00d-8845-4ea6-a6be-d97f67891d9b None None] MFA Rules not processed for user `95e9f085e4f04d118373b8abff27543a`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:41.665496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-eedda00d-8845-4ea6-a6be-d97f67891d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:41.795333 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 675/1367] 10.222.0.39 () {58 vars in 1059 bytes} [Fri Aug 5 19:55:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:55:41.803581 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd75d2-8410-4aa9-bbf5-bf43ad9c71cd None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:41.805248 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-c7bd75d2-8410-4aa9-bbf5-bf43ad9c71cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:41.966757 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd75d2-8410-4aa9-bbf5-bf43ad9c71cd None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:41.990285 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:42.096881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e09d808-a310-4756-9fd9-d7921b7e4713 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:42.097569 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09d808-a310-4756-9fd9-d7921b7e4713 service nova] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:42.097951 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09d808-a310-4756-9fd9-d7921b7e4713 service nova] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:42.098326 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09d808-a310-4756-9fd9-d7921b7e4713 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:42.100073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-0e09d808-a310-4756-9fd9-d7921b7e4713 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:42.202204 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e09d808-a310-4756-9fd9-d7921b7e4713 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:42.203953 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e09d808-a310-4756-9fd9-d7921b7e4713 service nova] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:42.227313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 693/1368] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:55:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:55:43.339387 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06557532-74da-4ee4-add1-09610ce315ef None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:43.340590 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06557532-74da-4ee4-add1-09610ce315ef None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:43.340854 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06557532-74da-4ee4-add1-09610ce315ef None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:43.342495 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 676/1369] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:55:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:55:43.347697 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806dad7b-ca79-46ee-8e0a-66e8c531869d None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:43.348173 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806dad7b-ca79-46ee-8e0a-66e8c531869d None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:43.348267 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806dad7b-ca79-46ee-8e0a-66e8c531869d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:43.392890 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:43.453365 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-806dad7b-ca79-46ee-8e0a-66e8c531869d None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:43.454828 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-806dad7b-ca79-46ee-8e0a-66e8c531869d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:43.566888 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 694/1370] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:55:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:55:43.891032 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99720cdf-3ac1-4949-9bd4-121076fdd54f None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:43.891466 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99720cdf-3ac1-4949-9bd4-121076fdd54f None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:43.891845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99720cdf-3ac1-4949-9bd4-121076fdd54f None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:43.893499 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 677/1371] 10.222.0.39 () {54 vars in 964 bytes} [Fri Aug 5 19:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:55:43.899158 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7518a-1292-4788-bdf8-66dd61074bdf None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:43.899450 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7518a-1292-4788-bdf8-66dd61074bdf None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:43.899726 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7518a-1292-4788-bdf8-66dd61074bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:43.997843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-f9e7518a-1292-4788-bdf8-66dd61074bdf None None] MFA Rules not processed for user `44dfe45766d743558ed82e464721cf72`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:43.999604 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-f9e7518a-1292-4788-bdf8-66dd61074bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:44.061253 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:44.103739 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 695/1372] 10.222.0.39 () {58 vars in 1061 bytes} [Fri Aug 5 19:55:43 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:55:44.122496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99720cdf-3ac1-4949-9bd4-121076fdd54f None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:44.124970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-99720cdf-3ac1-4949-9bd4-121076fdd54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:44.201637 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:44.268816 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99720cdf-3ac1-4949-9bd4-121076fdd54f None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:44.358960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3860e-1887-4451-a73b-2a8b8446f489 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:44.359639 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3860e-1887-4451-a73b-2a8b8446f489 service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:44.359907 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3860e-1887-4451-a73b-2a8b8446f489 service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:44.360120 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3860e-1887-4451-a73b-2a8b8446f489 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:44.361818 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-6be3860e-1887-4451-a73b-2a8b8446f489 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:44.472686 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:44.486144 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9b7f0-5576-4634-b214-fde4a2f23342 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:44.486664 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9b7f0-5576-4634-b214-fde4a2f23342 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:44.486664 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9b7f0-5576-4634-b214-fde4a2f23342 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:44.488205 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 696/1373] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:55:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:55:44.493466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20dcb6-4402-4b25-a12a-4134bf478591 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:44.493760 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20dcb6-4402-4b25-a12a-4134bf478591 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:44.494038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20dcb6-4402-4b25-a12a-4134bf478591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:44.494697 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3860e-1887-4451-a73b-2a8b8446f489 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:44.496142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3860e-1887-4451-a73b-2a8b8446f489 service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:44.519121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 678/1374] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:55:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:55:44.591659 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-1f20dcb6-4402-4b25-a12a-4134bf478591 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:44.592839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-1f20dcb6-4402-4b25-a12a-4134bf478591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:44.713113 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 697/1375] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:55:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:55:44.756610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3860e-1887-4451-a73b-2a8b8446f489 service cinder] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:44.757748 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-6be3860e-1887-4451-a73b-2a8b8446f489 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:44.875876 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3860e-1887-4451-a73b-2a8b8446f489 service cinder] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:44.982765 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:44.995408 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d920b9-8fce-4b4f-9a92-9281acc5ab0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:44.996017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d920b9-8fce-4b4f-9a92-9281acc5ab0a service nova] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:44.996873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d920b9-8fce-4b4f-9a92-9281acc5ab0a service nova] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:44.997144 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d920b9-8fce-4b4f-9a92-9281acc5ab0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:44.998765 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-a6d920b9-8fce-4b4f-9a92-9281acc5ab0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:45.089995 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d920b9-8fce-4b4f-9a92-9281acc5ab0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:45.091653 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d920b9-8fce-4b4f-9a92-9281acc5ab0a service nova] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:45.109733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 679/1376] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:55:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:55:47.212621 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ba56e-44a6-496d-aee9-e2a3420e4e45 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:47.213170 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ba56e-44a6-496d-aee9-e2a3420e4e45 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:47.213170 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ba56e-44a6-496d-aee9-e2a3420e4e45 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:47.218255 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 698/1377] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:55:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:55:47.222697 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b72d9-9b0c-4d04-8258-9a9aadb567dd None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:47.223135 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b72d9-9b0c-4d04-8258-9a9aadb567dd None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:47.225447 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b72d9-9b0c-4d04-8258-9a9aadb567dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:47.313753 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-b36b72d9-9b0c-4d04-8258-9a9aadb567dd None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:47.314892 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-b36b72d9-9b0c-4d04-8258-9a9aadb567dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:47.377776 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:47.421321 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 680/1378] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:55:47.466231 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ba56e-44a6-496d-aee9-e2a3420e4e45 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:47.475006 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-7f2ba56e-44a6-496d-aee9-e2a3420e4e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:47.520735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:47.620348 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ba56e-44a6-496d-aee9-e2a3420e4e45 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:47.780680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4076f-f388-497b-84f0-61c92c5723cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:47.781223 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4076f-f388-497b-84f0-61c92c5723cd service nova] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:47.781503 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4076f-f388-497b-84f0-61c92c5723cd service nova] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:47.781772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4076f-f388-497b-84f0-61c92c5723cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:47.783440 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-24d4076f-f388-497b-84f0-61c92c5723cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:47.795780 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:47.877849 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d4076f-f388-497b-84f0-61c92c5723cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:47.879460 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d4076f-f388-497b-84f0-61c92c5723cd service nova] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:47.898524 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 699/1379] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:55:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:55:50.021671 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fdbcf7-ade3-4b6e-be79-3c676c612d0b None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:50.022233 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fdbcf7-ade3-4b6e-be79-3c676c612d0b None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:50.022233 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fdbcf7-ade3-4b6e-be79-3c676c612d0b None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:50.023880 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 681/1380] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:55:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:55:50.029490 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a16222-e0c1-423a-8b11-525f7ba852a5 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:50.029926 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a16222-e0c1-423a-8b11-525f7ba852a5 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:50.030038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a16222-e0c1-423a-8b11-525f7ba852a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:50.106934 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-26a16222-e0c1-423a-8b11-525f7ba852a5 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:50.108387 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-26a16222-e0c1-423a-8b11-525f7ba852a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:50.206602 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:50.221120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 700/1381] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:55:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:55:50.266715 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fdbcf7-ade3-4b6e-be79-3c676c612d0b None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:50.268361 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-01fdbcf7-ade3-4b6e-be79-3c676c612d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:50.440291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fdbcf7-ade3-4b6e-be79-3c676c612d0b None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:50.492731 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:50.566038 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e887c97e-eb54-45a2-9ac2-55f71cf4b009 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:50.566800 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887c97e-eb54-45a2-9ac2-55f71cf4b009 service nova] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:50.567208 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887c97e-eb54-45a2-9ac2-55f71cf4b009 service nova] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:50.567571 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887c97e-eb54-45a2-9ac2-55f71cf4b009 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:50.569310 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-e887c97e-eb54-45a2-9ac2-55f71cf4b009 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:50.678044 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e887c97e-eb54-45a2-9ac2-55f71cf4b009 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:50.679797 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e887c97e-eb54-45a2-9ac2-55f71cf4b009 service nova] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:50.701052 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 682/1382] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:55:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:55:52.795391 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187db2c4-04ed-4bd5-ab1a-9eeb32c2c600 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:52.795951 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187db2c4-04ed-4bd5-ab1a-9eeb32c2c600 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:52.795951 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187db2c4-04ed-4bd5-ab1a-9eeb32c2c600 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:52.797338 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 701/1383] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:55:52.802656 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e2d4f-569c-474c-9b2d-86984d5f02e2 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:52.803091 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e2d4f-569c-474c-9b2d-86984d5f02e2 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:52.803215 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e2d4f-569c-474c-9b2d-86984d5f02e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:52.874438 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-ab8e2d4f-569c-474c-9b2d-86984d5f02e2 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:52.875697 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-ab8e2d4f-569c-474c-9b2d-86984d5f02e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:52.877855 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:52.982187 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 683/1384] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:55:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:55:53.024408 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187db2c4-04ed-4bd5-ab1a-9eeb32c2c600 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:53.025977 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-187db2c4-04ed-4bd5-ab1a-9eeb32c2c600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:53.217466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187db2c4-04ed-4bd5-ab1a-9eeb32c2c600 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:53.296191 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:53.364534 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90149998-9318-4563-8fdb-7c9ee43540be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:53.365209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90149998-9318-4563-8fdb-7c9ee43540be service nova] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:53.365490 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90149998-9318-4563-8fdb-7c9ee43540be service nova] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:53.365766 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90149998-9318-4563-8fdb-7c9ee43540be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:53.367415 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-90149998-9318-4563-8fdb-7c9ee43540be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:53.463848 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90149998-9318-4563-8fdb-7c9ee43540be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:53.465942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90149998-9318-4563-8fdb-7c9ee43540be service nova] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:53.486066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 702/1385] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:55:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:55:55.518163 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b0eb7-efbd-4b1e-9318-769b5f01296e None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:55.518795 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b0eb7-efbd-4b1e-9318-769b5f01296e None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:55.518795 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b0eb7-efbd-4b1e-9318-769b5f01296e None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:55.520923 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 684/1386] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:55:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:55:55.526249 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92590b-7742-40ba-9f9c-6b60c3d6d212 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:55.526708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92590b-7742-40ba-9f9c-6b60c3d6d212 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:55.526825 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92590b-7742-40ba-9f9c-6b60c3d6d212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:55.603532 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-bb92590b-7742-40ba-9f9c-6b60c3d6d212 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:55.604789 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-bb92590b-7742-40ba-9f9c-6b60c3d6d212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:55.638571 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:55.726561 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 703/1387] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:55:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:55:55.783360 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b0eb7-efbd-4b1e-9318-769b5f01296e None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:55.785392 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-415b0eb7-efbd-4b1e-9318-769b5f01296e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:55.942746 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:55.961191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b0eb7-efbd-4b1e-9318-769b5f01296e None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:56.082359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcda2e0-c7ab-4882-8888-02883d41e35d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:56.083144 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcda2e0-c7ab-4882-8888-02883d41e35d service nova] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:56.083450 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcda2e0-c7ab-4882-8888-02883d41e35d service nova] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:56.083825 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcda2e0-c7ab-4882-8888-02883d41e35d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:56.085638 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-3fcda2e0-c7ab-4882-8888-02883d41e35d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:56.190489 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcda2e0-c7ab-4882-8888-02883d41e35d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:56.192056 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcda2e0-c7ab-4882-8888-02883d41e35d service nova] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:56.197190 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:56.211327 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 685/1388] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:55:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:55:58.337188 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5fdcb-f11b-47a2-aa1e-71dc7ef5b542 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:58.337763 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5fdcb-f11b-47a2-aa1e-71dc7ef5b542 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:58.337869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5fdcb-f11b-47a2-aa1e-71dc7ef5b542 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:58.339170 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 704/1389] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:55:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:55:58.345410 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa18875-e9c7-454b-aec9-9acc36782b28 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:58.345868 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa18875-e9c7-454b-aec9-9acc36782b28 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:58.345966 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa18875-e9c7-454b-aec9-9acc36782b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:58.420107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-1aa18875-e9c7-454b-aec9-9acc36782b28 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:58.421236 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-1aa18875-e9c7-454b-aec9-9acc36782b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:58.530324 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 686/1390] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:55:58.576459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5fdcb-f11b-47a2-aa1e-71dc7ef5b542 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:58.578482 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-aae5fdcb-f11b-47a2-aa1e-71dc7ef5b542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:58.739530 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5fdcb-f11b-47a2-aa1e-71dc7ef5b542 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:58.750499 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:58.865389 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08b840-ab23-4f2b-8e0f-7629dd9f2ba1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:58.866045 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08b840-ab23-4f2b-8e0f-7629dd9f2ba1 service nova] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:58.866293 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08b840-ab23-4f2b-8e0f-7629dd9f2ba1 service nova] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:58.866566 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08b840-ab23-4f2b-8e0f-7629dd9f2ba1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:58.868181 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-ee08b840-ab23-4f2b-8e0f-7629dd9f2ba1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:58.962796 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee08b840-ab23-4f2b-8e0f-7629dd9f2ba1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:58.964434 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee08b840-ab23-4f2b-8e0f-7629dd9f2ba1 service nova] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:58.983650 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 705/1391] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:55:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:55:59.924238 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117b6c3-6374-46bf-8969-e0f9d4803359 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:59.924864 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117b6c3-6374-46bf-8969-e0f9d4803359 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:59.924864 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117b6c3-6374-46bf-8969-e0f9d4803359 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:59.926307 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 687/1392] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:55:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:55:59.931416 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c070f5-2976-4acd-b120-257e13aa3e47 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:59.931862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c070f5-2976-4acd-b120-257e13aa3e47 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:59.931970 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c070f5-2976-4acd-b120-257e13aa3e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:59.948679 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:00.003676 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-21c070f5-2976-4acd-b120-257e13aa3e47 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:00.004920 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-21c070f5-2976-4acd-b120-257e13aa3e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:00.110113 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 706/1393] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:00.172001 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacee2d-fa5e-413b-8489-ebc61a9ed507 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:00.172294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacee2d-fa5e-413b-8489-ebc61a9ed507 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:00.172569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacee2d-fa5e-413b-8489-ebc61a9ed507 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:00.173976 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 688/1394] 10.222.0.39 () {54 vars in 966 bytes} [Fri Aug 5 19:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:56:00.179309 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538693f-cac7-42a3-85c1-f719f1ded939 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:00.179608 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538693f-cac7-42a3-85c1-f719f1ded939 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:00.180056 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538693f-cac7-42a3-85c1-f719f1ded939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:00.271364 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-7538693f-cac7-42a3-85c1-f719f1ded939 None None] MFA Rules not processed for user `2fe69b398c1f4e18b625a63ae40b01e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:00.272796 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-7538693f-cac7-42a3-85c1-f719f1ded939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:00.322783 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:00.365996 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 707/1395] 10.222.0.39 () {58 vars in 1063 bytes} [Fri Aug 5 19:56:00 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:00.377496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aacee2d-fa5e-413b-8489-ebc61a9ed507 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:00.379074 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-2aacee2d-fa5e-413b-8489-ebc61a9ed507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:00.464433 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:00.503981 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aacee2d-fa5e-413b-8489-ebc61a9ed507 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:00.587307 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86f77c-5441-489d-9a92-dc353272b82d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe69b398c1f4e18b625a63ae40b01e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:00.588029 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86f77c-5441-489d-9a92-dc353272b82d service glance] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:00.588433 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86f77c-5441-489d-9a92-dc353272b82d service glance] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:00.588918 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86f77c-5441-489d-9a92-dc353272b82d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:00.590738 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-bf86f77c-5441-489d-9a92-dc353272b82d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:00.743738 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86f77c-5441-489d-9a92-dc353272b82d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:00.746421 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86f77c-5441-489d-9a92-dc353272b82d service glance] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:00.771193 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:00.775683 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 689/1396] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:56:01.221957 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a4de6c-9455-4188-9dbd-c911cb77c4a7 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:01.222570 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a4de6c-9455-4188-9dbd-c911cb77c4a7 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:01.223078 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a4de6c-9455-4188-9dbd-c911cb77c4a7 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:01.225098 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 708/1397] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:56:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:56:01.231484 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec26a7-defe-44b9-9c84-a19c4c8cf7e7 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:01.231986 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec26a7-defe-44b9-9c84-a19c4c8cf7e7 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:01.232401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec26a7-defe-44b9-9c84-a19c4c8cf7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:01.323495 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-eeec26a7-defe-44b9-9c84-a19c4c8cf7e7 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:01.325951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-eeec26a7-defe-44b9-9c84-a19c4c8cf7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:01.443737 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 690/1398] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:56:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:01.491251 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a4de6c-9455-4188-9dbd-c911cb77c4a7 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:01.492854 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-a0a4de6c-9455-4188-9dbd-c911cb77c4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:01.637781 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411bb83-f106-438f-a0ab-a588629178d7 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:01.638385 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411bb83-f106-438f-a0ab-a588629178d7 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:01.638704 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411bb83-f106-438f-a0ab-a588629178d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:01.667253 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a4de6c-9455-4188-9dbd-c911cb77c4a7 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:01.709525 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-0411bb83-f106-438f-a0ab-a588629178d7 None None] MFA Rules not processed for user `3488a7d2258242c0859a105c1d162767`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:01.710961 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-0411bb83-f106-438f-a0ab-a588629178d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:01.716540 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:01.743504 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:01.796357 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f0284-ac80-4a67-b8cc-0bc35da92a7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:01.797007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f0284-ac80-4a67-b8cc-0bc35da92a7d service nova] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:01.797290 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f0284-ac80-4a67-b8cc-0bc35da92a7d service nova] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:01.797564 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f0284-ac80-4a67-b8cc-0bc35da92a7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:01.799216 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-261f0284-ac80-4a67-b8cc-0bc35da92a7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:01.805674 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 691/1399] 10.222.0.39 () {58 vars in 1009 bytes} [Fri Aug 5 19:56:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:56:01.859512 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcaaad8-2fd3-4dbc-9d7a-a93b44f366ed None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:01.859797 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcaaad8-2fd3-4dbc-9d7a-a93b44f366ed None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:01.860063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcaaad8-2fd3-4dbc-9d7a-a93b44f366ed None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:01.861483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 692/1400] 10.222.0.39 () {54 vars in 943 bytes} [Fri Aug 5 19:56:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:56:01.867760 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a34338-0f5d-462d-8d6c-d6d1f80881b5 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:01.868156 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a34338-0f5d-462d-8d6c-d6d1f80881b5 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:01.868446 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a34338-0f5d-462d-8d6c-d6d1f80881b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:01.913470 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261f0284-ac80-4a67-b8cc-0bc35da92a7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:01.915144 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261f0284-ac80-4a67-b8cc-0bc35da92a7d service nova] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:01.934278 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 709/1401] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:56:01.954475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-93a34338-0f5d-462d-8d6c-d6d1f80881b5 None None] MFA Rules not processed for user `91d3d43cb0bd4ca3b8e8f10e29cc421c`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:01.955928 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-93a34338-0f5d-462d-8d6c-d6d1f80881b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:02.094344 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 693/1402] 10.222.0.39 () {58 vars in 1040 bytes} [Fri Aug 5 19:56:01 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:56:02.104373 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f0284-ac80-4a67-b8cc-0bc35da92a7d service nova] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:02.106517 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-261f0284-ac80-4a67-b8cc-0bc35da92a7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:02.165186 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:02.212916 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f0284-ac80-4a67-b8cc-0bc35da92a7d service nova] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:02.321456 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3153b6-7a91-482a-8aee-2adcbcbb0b4f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91d3d43cb0bd4ca3b8e8f10e29cc421c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:02.322179 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3153b6-7a91-482a-8aee-2adcbcbb0b4f service swift] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:02.322348 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3153b6-7a91-482a-8aee-2adcbcbb0b4f service swift] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:02.322623 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3153b6-7a91-482a-8aee-2adcbcbb0b4f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:02.324998 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-ad3153b6-7a91-482a-8aee-2adcbcbb0b4f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:02.409560 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:02.413819 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3153b6-7a91-482a-8aee-2adcbcbb0b4f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:02.424486 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3153b6-7a91-482a-8aee-2adcbcbb0b4f service swift] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:02.433454 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 710/1403] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Aug 5 19:56:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 19:56:04.086020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc399d4f-745c-45d9-8863-01c218c2ca90 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:04.087362 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc399d4f-745c-45d9-8863-01c218c2ca90 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:04.087675 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc399d4f-745c-45d9-8863-01c218c2ca90 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:04.089390 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 694/1404] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:56:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:56:04.096479 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261db884-569b-4cbe-98be-9cb37cb701ad None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:04.096479 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261db884-569b-4cbe-98be-9cb37cb701ad None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:04.096479 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261db884-569b-4cbe-98be-9cb37cb701ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:04.190034 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-261db884-569b-4cbe-98be-9cb37cb701ad None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:04.191311 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-261db884-569b-4cbe-98be-9cb37cb701ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:04.317156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 711/1405] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:56:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:04.363120 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc399d4f-745c-45d9-8863-01c218c2ca90 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:04.364676 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-dc399d4f-745c-45d9-8863-01c218c2ca90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:04.375832 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:04.580511 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc399d4f-745c-45d9-8863-01c218c2ca90 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:04.673705 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:04.718916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd047e51-5ff2-49eb-843b-2556a30ee8b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:04.719561 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd047e51-5ff2-49eb-843b-2556a30ee8b7 service nova] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:04.719836 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd047e51-5ff2-49eb-843b-2556a30ee8b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:04.720101 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd047e51-5ff2-49eb-843b-2556a30ee8b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:04.721776 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-cd047e51-5ff2-49eb-843b-2556a30ee8b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:04.822441 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd047e51-5ff2-49eb-843b-2556a30ee8b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:04.824054 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd047e51-5ff2-49eb-843b-2556a30ee8b7 service nova] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:04.845232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 695/1406] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:56:06.944660 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66531e89-b75e-4f25-9414-4cec59c1f0b4 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:06.945260 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66531e89-b75e-4f25-9414-4cec59c1f0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:06.945260 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66531e89-b75e-4f25-9414-4cec59c1f0b4 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:06.946576 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 712/1407] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:56:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:56:06.952569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2f973-0070-4476-88ce-c49ac8b845f7 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:06.953081 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2f973-0070-4476-88ce-c49ac8b845f7 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:06.953177 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2f973-0070-4476-88ce-c49ac8b845f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:07.035169 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-49d2f973-0070-4476-88ce-c49ac8b845f7 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:07.036398 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-49d2f973-0070-4476-88ce-c49ac8b845f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:07.083336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:07.154970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 696/1408] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:07.202558 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66531e89-b75e-4f25-9414-4cec59c1f0b4 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:07.204093 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-66531e89-b75e-4f25-9414-4cec59c1f0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:07.291892 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:07.376569 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66531e89-b75e-4f25-9414-4cec59c1f0b4 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:07.514872 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7967ab8f-ba18-417c-91ec-d16fa11b5542 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:07.515508 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967ab8f-ba18-417c-91ec-d16fa11b5542 service nova] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:07.515848 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967ab8f-ba18-417c-91ec-d16fa11b5542 service nova] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:07.516192 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967ab8f-ba18-417c-91ec-d16fa11b5542 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:07.517922 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-7967ab8f-ba18-417c-91ec-d16fa11b5542 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:07.562715 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:07.631841 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7967ab8f-ba18-417c-91ec-d16fa11b5542 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:07.633621 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7967ab8f-ba18-417c-91ec-d16fa11b5542 service nova] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:07.655088 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 713/1409] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:56:17.152960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=26 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:17.652505 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=30 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:25.489124 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb265234-3922-4a9f-b1b7-fada4d304d7b None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:25.489678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb265234-3922-4a9f-b1b7-fada4d304d7b None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:25.489678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb265234-3922-4a9f-b1b7-fada4d304d7b None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:25.490978 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 697/1410] 10.222.0.39 () {54 vars in 918 bytes} [Fri Aug 5 19:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:56:25.496367 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a22010-3852-4156-9b5e-cf9bc96d5f51 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:25.496854 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a22010-3852-4156-9b5e-cf9bc96d5f51 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:25.496967 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a22010-3852-4156-9b5e-cf9bc96d5f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:25.575778 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-74a22010-3852-4156-9b5e-cf9bc96d5f51 None None] MFA Rules not processed for user `c61e2c954f9a49abad9f3de17ee39f63`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:25.577035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-74a22010-3852-4156-9b5e-cf9bc96d5f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:25.687685 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 714/1411] 10.222.0.39 () {58 vars in 1015 bytes} [Fri Aug 5 19:56:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:25.703414 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb6bf9-4949-43db-812b-70318aacf16e None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:25.703699 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb6bf9-4949-43db-812b-70318aacf16e None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:25.703973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb6bf9-4949-43db-812b-70318aacf16e None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:25.705386 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 698/1412] 10.222.0.39 () {54 vars in 918 bytes} [Fri Aug 5 19:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:56:25.710038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15429042-c1b6-4fa9-9de5-eddfc7d54821 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:25.710326 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15429042-c1b6-4fa9-9de5-eddfc7d54821 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:25.710601 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15429042-c1b6-4fa9-9de5-eddfc7d54821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:25.785055 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:25.788417 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-15429042-c1b6-4fa9-9de5-eddfc7d54821 None None] MFA Rules not processed for user `95e9f085e4f04d118373b8abff27543a`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:25.790635 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-15429042-c1b6-4fa9-9de5-eddfc7d54821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:25.938711 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 715/1413] 10.222.0.39 () {58 vars in 1015 bytes} [Fri Aug 5 19:56:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:56:28.508948 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4a1b6-cc1f-4b05-ad75-e4e9a08188af None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:28.509484 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4a1b6-cc1f-4b05-ad75-e4e9a08188af None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:28.509484 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4a1b6-cc1f-4b05-ad75-e4e9a08188af None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:28.510850 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 699/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 19:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:56:28.516174 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d251030-64de-4c22-920a-8e6d73013a7b None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:28.516657 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d251030-64de-4c22-920a-8e6d73013a7b None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:28.516766 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d251030-64de-4c22-920a-8e6d73013a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:28.585503 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-0d251030-64de-4c22-920a-8e6d73013a7b None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:28.586712 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-0d251030-64de-4c22-920a-8e6d73013a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:28.644100 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:28.688157 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 716/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 19:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:28.731658 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd4a1b6-cc1f-4b05-ad75-e4e9a08188af None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:28.733246 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-4fd4a1b6-cc1f-4b05-ad75-e4e9a08188af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:28.881331 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd4a1b6-cc1f-4b05-ad75-e4e9a08188af None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:28.956168 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:28.998420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2a5494-2e33-45ea-a317-ef753497da26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:28.999033 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a5494-2e33-45ea-a317-ef753497da26 service nova] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:28.999314 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a5494-2e33-45ea-a317-ef753497da26 service nova] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:28.999589 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a5494-2e33-45ea-a317-ef753497da26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:29.001260 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-4a2a5494-2e33-45ea-a317-ef753497da26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:29.093887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2a5494-2e33-45ea-a317-ef753497da26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:29.095476 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2a5494-2e33-45ea-a317-ef753497da26 service nova] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:29.114173 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 700/1416] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:56:30.032448 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369ecd1-96b5-49ff-b8b8-15d0d767bebc None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:30.033111 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369ecd1-96b5-49ff-b8b8-15d0d767bebc None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:30.033111 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369ecd1-96b5-49ff-b8b8-15d0d767bebc None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:30.034425 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 717/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 19:56:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:56:30.039470 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81601033-a9cc-4f75-92e0-87c19c1a4204 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:30.039939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81601033-a9cc-4f75-92e0-87c19c1a4204 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:30.040053 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81601033-a9cc-4f75-92e0-87c19c1a4204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:30.241269 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-81601033-a9cc-4f75-92e0-87c19c1a4204 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:30.242488 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-81601033-a9cc-4f75-92e0-87c19c1a4204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:30.271937 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:30.351280 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 701/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 19:56:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:30.506492 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5369ecd1-96b5-49ff-b8b8-15d0d767bebc None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:30.508063 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5369ecd1-96b5-49ff-b8b8-15d0d767bebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:30.658731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5369ecd1-96b5-49ff-b8b8-15d0d767bebc None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:30.701316 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:30.778612 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6205a75-0532-41f3-aaf4-5847b6e941c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:30.779230 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6205a75-0532-41f3-aaf4-5847b6e941c1 service nova] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:30.779505 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6205a75-0532-41f3-aaf4-5847b6e941c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:30.779771 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6205a75-0532-41f3-aaf4-5847b6e941c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:30.781409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e6205a75-0532-41f3-aaf4-5847b6e941c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:30.871597 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6205a75-0532-41f3-aaf4-5847b6e941c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:30.873342 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6205a75-0532-41f3-aaf4-5847b6e941c1 service nova] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:30.892488 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 718/1419] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:56:31.795402 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf75cd4a-a07c-4110-8be2-c9934452fa26 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:31.795992 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf75cd4a-a07c-4110-8be2-c9934452fa26 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:31.795992 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf75cd4a-a07c-4110-8be2-c9934452fa26 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:31.797352 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 702/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 19:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:56:31.802172 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96817aab-054d-4c2b-9d2e-4c00ddeb2c70 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:31.802583 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96817aab-054d-4c2b-9d2e-4c00ddeb2c70 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:31.802692 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96817aab-054d-4c2b-9d2e-4c00ddeb2c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:31.858369 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:31.882192 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-96817aab-054d-4c2b-9d2e-4c00ddeb2c70 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:31.883406 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-96817aab-054d-4c2b-9d2e-4c00ddeb2c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:31.981390 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 719/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 19:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:32.131439 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf75cd4a-a07c-4110-8be2-c9934452fa26 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:32.132995 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-cf75cd4a-a07c-4110-8be2-c9934452fa26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:32.282365 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf75cd4a-a07c-4110-8be2-c9934452fa26 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:32.294993 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:32.398573 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4318233a-273b-40c1-abd3-e4c7ca17335b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:32.399088 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318233a-273b-40c1-abd3-e4c7ca17335b service nova] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:32.399362 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318233a-273b-40c1-abd3-e4c7ca17335b service nova] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:32.399629 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318233a-273b-40c1-abd3-e4c7ca17335b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:32.401260 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-4318233a-273b-40c1-abd3-e4c7ca17335b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:32.490688 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4318233a-273b-40c1-abd3-e4c7ca17335b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:32.492277 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4318233a-273b-40c1-abd3-e4c7ca17335b service nova] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:32.511202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 703/1422] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:56:33.380065 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c29f1-7fe5-4b60-ae8a-e0ad620231bd None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:33.380624 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c29f1-7fe5-4b60-ae8a-e0ad620231bd None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:33.380624 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c29f1-7fe5-4b60-ae8a-e0ad620231bd None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:33.381999 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 720/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 19:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:56:33.387682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24870d28-d6e2-4df0-922f-738a8042da24 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:33.388126 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24870d28-d6e2-4df0-922f-738a8042da24 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:33.388231 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24870d28-d6e2-4df0-922f-738a8042da24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:33.407986 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:33.465121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-24870d28-d6e2-4df0-922f-738a8042da24 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:33.466341 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-24870d28-d6e2-4df0-922f-738a8042da24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:33.563475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 704/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 19:56:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:33.713877 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c29f1-7fe5-4b60-ae8a-e0ad620231bd None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:33.715316 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5a4c29f1-7fe5-4b60-ae8a-e0ad620231bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:33.829872 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:33.865566 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c29f1-7fe5-4b60-ae8a-e0ad620231bd None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:33.978750 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338cf89-355c-4d9b-92b9-68d64343a097 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:33.979280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338cf89-355c-4d9b-92b9-68d64343a097 service nova] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:33.979558 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338cf89-355c-4d9b-92b9-68d64343a097 service nova] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:33.979830 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338cf89-355c-4d9b-92b9-68d64343a097 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:33.981834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-6338cf89-355c-4d9b-92b9-68d64343a097 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:34.063772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:34.078320 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338cf89-355c-4d9b-92b9-68d64343a097 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:34.079916 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338cf89-355c-4d9b-92b9-68d64343a097 service nova] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:34.098323 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 721/1425] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:56:34.991970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005578a-85d8-4d53-a058-63b1a2e1ea04 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:34.992550 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005578a-85d8-4d53-a058-63b1a2e1ea04 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:34.992550 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005578a-85d8-4d53-a058-63b1a2e1ea04 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:34.994303 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 705/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 19:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:56:34.999853 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21dfbb-d30f-4425-81ed-90a211adcc15 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:35.000317 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21dfbb-d30f-4425-81ed-90a211adcc15 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:35.000426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21dfbb-d30f-4425-81ed-90a211adcc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:35.080861 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-6f21dfbb-d30f-4425-81ed-90a211adcc15 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:35.082067 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-6f21dfbb-d30f-4425-81ed-90a211adcc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:35.181289 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 722/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 19:56:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:35.342591 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005578a-85d8-4d53-a058-63b1a2e1ea04 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:35.344009 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-0005578a-85d8-4d53-a058-63b1a2e1ea04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:35.430402 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:35.492459 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005578a-85d8-4d53-a058-63b1a2e1ea04 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:35.609745 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c530bb1-f2a5-41f6-b48d-b55f1edf770e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:35.610251 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c530bb1-f2a5-41f6-b48d-b55f1edf770e service nova] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:35.610529 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c530bb1-f2a5-41f6-b48d-b55f1edf770e service nova] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:35.610800 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c530bb1-f2a5-41f6-b48d-b55f1edf770e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:35.612407 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-1c530bb1-f2a5-41f6-b48d-b55f1edf770e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:35.663417 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:35.705493 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c530bb1-f2a5-41f6-b48d-b55f1edf770e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:35.707056 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c530bb1-f2a5-41f6-b48d-b55f1edf770e service nova] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:35.725990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 706/1428] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:56:36.553340 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574fd1a-cf70-4e96-8b51-cc2aa02507f7 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:36.553912 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574fd1a-cf70-4e96-8b51-cc2aa02507f7 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:36.553912 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574fd1a-cf70-4e96-8b51-cc2aa02507f7 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:36.555193 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 723/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 19:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:56:36.561286 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3e7a9-9446-4afd-806f-74b981e39e3a None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:36.561762 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3e7a9-9446-4afd-806f-74b981e39e3a None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:36.561864 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3e7a9-9446-4afd-806f-74b981e39e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:36.654418 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-84f3e7a9-9446-4afd-806f-74b981e39e3a None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:36.655635 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-84f3e7a9-9446-4afd-806f-74b981e39e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:36.752344 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 707/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 19:56:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:36.904851 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574fd1a-cf70-4e96-8b51-cc2aa02507f7 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:36.906403 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-7574fd1a-cf70-4e96-8b51-cc2aa02507f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:36.954371 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:37.052504 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574fd1a-cf70-4e96-8b51-cc2aa02507f7 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:37.166888 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a0a2f-d598-4ee8-89e5-a855bf3df9bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:37.167609 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a0a2f-d598-4ee8-89e5-a855bf3df9bc service nova] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:37.167893 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a0a2f-d598-4ee8-89e5-a855bf3df9bc service nova] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:37.168165 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a0a2f-d598-4ee8-89e5-a855bf3df9bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:37.169816 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-4f0a0a2f-d598-4ee8-89e5-a855bf3df9bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:37.176912 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:37.262849 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a0a2f-d598-4ee8-89e5-a855bf3df9bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:37.264450 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a0a2f-d598-4ee8-89e5-a855bf3df9bc service nova] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:37.283949 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 724/1431] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:56:38.097983 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6300b-818f-42b5-9958-56067e2921ca None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:38.098547 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6300b-818f-42b5-9958-56067e2921ca None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:38.098547 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6300b-818f-42b5-9958-56067e2921ca None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:38.099923 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 708/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 19:56:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:56:38.104834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f3419-27ba-4dbe-9e3c-00c819f4103d None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:38.105911 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f3419-27ba-4dbe-9e3c-00c819f4103d None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:38.106013 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f3419-27ba-4dbe-9e3c-00c819f4103d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:38.184774 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-bf6f3419-27ba-4dbe-9e3c-00c819f4103d None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:38.185989 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-bf6f3419-27ba-4dbe-9e3c-00c819f4103d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:38.253729 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:38.283228 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 725/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 19:56:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:38.436869 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc6300b-818f-42b5-9958-56067e2921ca None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:38.438415 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-fcc6300b-818f-42b5-9958-56067e2921ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:38.456674 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:38.586715 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc6300b-818f-42b5-9958-56067e2921ca None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:38.694875 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:38.722618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e9a7d-8809-4eae-98a0-8ea2e99a2910 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:38.723224 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e9a7d-8809-4eae-98a0-8ea2e99a2910 service nova] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:38.723498 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e9a7d-8809-4eae-98a0-8ea2e99a2910 service nova] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:38.723765 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e9a7d-8809-4eae-98a0-8ea2e99a2910 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:38.725404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-126e9a7d-8809-4eae-98a0-8ea2e99a2910 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:38.814942 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126e9a7d-8809-4eae-98a0-8ea2e99a2910 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:38.816507 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126e9a7d-8809-4eae-98a0-8ea2e99a2910 service nova] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:38.834605 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 709/1434] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:56:39.648904 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e1a03-96e3-491a-99c0-54f9ee8c3eea None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:39.649522 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e1a03-96e3-491a-99c0-54f9ee8c3eea None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:39.649522 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e1a03-96e3-491a-99c0-54f9ee8c3eea None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:39.650823 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 726/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 19:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:56:39.656679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef735d2d-6830-490b-9232-648d48e8788c None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:39.657168 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef735d2d-6830-490b-9232-648d48e8788c None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:39.657279 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef735d2d-6830-490b-9232-648d48e8788c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:39.733771 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-ef735d2d-6830-490b-9232-648d48e8788c None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:39.734982 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-ef735d2d-6830-490b-9232-648d48e8788c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:39.776807 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:39.833517 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 710/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 19:56:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:39.991972 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e1a03-96e3-491a-99c0-54f9ee8c3eea None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:39.993564 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-ba5e1a03-96e3-491a-99c0-54f9ee8c3eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:40.140738 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e1a03-96e3-491a-99c0-54f9ee8c3eea None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:40.196166 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:40.258542 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c3722a-019a-4414-bb27-110a1ec3e484 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:40.259151 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3722a-019a-4414-bb27-110a1ec3e484 service nova] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:40.259427 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3722a-019a-4414-bb27-110a1ec3e484 service nova] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:40.259697 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3722a-019a-4414-bb27-110a1ec3e484 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:40.261885 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-14c3722a-019a-4414-bb27-110a1ec3e484 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:40.353906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c3722a-019a-4414-bb27-110a1ec3e484 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:40.355602 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c3722a-019a-4414-bb27-110a1ec3e484 service nova] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:40.374853 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 727/1437] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:56:41.182051 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbde0ed-b9f2-4277-99d2-7e1be89889cf None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:41.182510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbde0ed-b9f2-4277-99d2-7e1be89889cf None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:41.182613 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbde0ed-b9f2-4277-99d2-7e1be89889cf None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:41.184364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 711/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 19:56:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:56:41.189295 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc1a1c-0ca3-46e1-be21-aab496dc87be None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:41.189734 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc1a1c-0ca3-46e1-be21-aab496dc87be None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:41.189856 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc1a1c-0ca3-46e1-be21-aab496dc87be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:41.267122 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-42cc1a1c-0ca3-46e1-be21-aab496dc87be None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:41.268310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-42cc1a1c-0ca3-46e1-be21-aab496dc87be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:41.270368 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:41.371181 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 728/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 19:56:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:56:41.521052 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbde0ed-b9f2-4277-99d2-7e1be89889cf None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:41.522883 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-7fbde0ed-b9f2-4277-99d2-7e1be89889cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:41.674911 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbde0ed-b9f2-4277-99d2-7e1be89889cf None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:41.701886 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:41.789634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf4a693-7e24-4c51-baa3-aeb7704910e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:41.790153 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4a693-7e24-4c51-baa3-aeb7704910e2 service nova] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:41.790429 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4a693-7e24-4c51-baa3-aeb7704910e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:41.790727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4a693-7e24-4c51-baa3-aeb7704910e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:41.792437 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-0cf4a693-7e24-4c51-baa3-aeb7704910e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:41.883723 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf4a693-7e24-4c51-baa3-aeb7704910e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:41.885335 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf4a693-7e24-4c51-baa3-aeb7704910e2 service nova] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:41.904597 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 712/1440] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:56:43.827743 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8e2fc-04df-410a-bde9-b0e38183ceb0 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:43.828319 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8e2fc-04df-410a-bde9-b0e38183ceb0 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:43.828319 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8e2fc-04df-410a-bde9-b0e38183ceb0 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:43.829672 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 729/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 19:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:56:43.835006 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca645697-8993-4c0d-b0c9-413360cb05a2 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:43.835439 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca645697-8993-4c0d-b0c9-413360cb05a2 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:43.835548 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca645697-8993-4c0d-b0c9-413360cb05a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:43.876912 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:43.903061 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-ca645697-8993-4c0d-b0c9-413360cb05a2 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:43.904282 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-ca645697-8993-4c0d-b0c9-413360cb05a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:43.998876 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 713/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 19:56:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:44.038111 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8e2fc-04df-410a-bde9-b0e38183ceb0 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:44.039651 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-1fc8e2fc-04df-410a-bde9-b0e38183ceb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:44.145873 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8e2fc-04df-410a-bde9-b0e38183ceb0 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:44.178554 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:44.220695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fffb075-98b2-45a3-95b0-8366a9e56fda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:44.221310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fffb075-98b2-45a3-95b0-8366a9e56fda service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:44.221597 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fffb075-98b2-45a3-95b0-8366a9e56fda service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:44.221869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fffb075-98b2-45a3-95b0-8366a9e56fda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:44.223489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-8fffb075-98b2-45a3-95b0-8366a9e56fda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:44.333710 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fffb075-98b2-45a3-95b0-8366a9e56fda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:44.335148 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fffb075-98b2-45a3-95b0-8366a9e56fda service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:44.354811 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 730/1443] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:56:46.540181 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7abe1-d127-4cc8-a8f9-d39dbf68b56d None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:46.540735 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7abe1-d127-4cc8-a8f9-d39dbf68b56d None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:46.540735 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7abe1-d127-4cc8-a8f9-d39dbf68b56d None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:46.542039 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 714/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 19:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:56:46.548046 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2521bb5-e224-46cf-9d9d-097f7428e8ad None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:46.548484 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2521bb5-e224-46cf-9d9d-097f7428e8ad None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:46.548610 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2521bb5-e224-46cf-9d9d-097f7428e8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:46.615809 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-c2521bb5-e224-46cf-9d9d-097f7428e8ad None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:46.617052 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-c2521bb5-e224-46cf-9d9d-097f7428e8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:46.631748 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:46.723978 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 731/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 19:56:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:46.763586 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b7abe1-d127-4cc8-a8f9-d39dbf68b56d None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:46.765133 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-81b7abe1-d127-4cc8-a8f9-d39dbf68b56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:46.869306 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b7abe1-d127-4cc8-a8f9-d39dbf68b56d None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:46.881793 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:46.945625 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4ebc9-3ad0-48af-b6f2-4636aad24bae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:46.946241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4ebc9-3ad0-48af-b6f2-4636aad24bae service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:46.946524 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4ebc9-3ad0-48af-b6f2-4636aad24bae service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:46.946795 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4ebc9-3ad0-48af-b6f2-4636aad24bae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:46.948468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-37a4ebc9-3ad0-48af-b6f2-4636aad24bae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:47.059124 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4ebc9-3ad0-48af-b6f2-4636aad24bae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:47.060550 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4ebc9-3ad0-48af-b6f2-4636aad24bae service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:47.079368 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 715/1446] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:56:49.433649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceefc7ae-069f-4cfe-becd-df607e8c9cb7 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:49.434300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceefc7ae-069f-4cfe-becd-df607e8c9cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:49.434300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceefc7ae-069f-4cfe-becd-df607e8c9cb7 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:49.435653 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 732/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 19:56:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:56:49.441194 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77362071-809d-4b48-8bc5-3fb157ae7e1a None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:49.441647 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77362071-809d-4b48-8bc5-3fb157ae7e1a None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:49.441753 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77362071-809d-4b48-8bc5-3fb157ae7e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:49.491706 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:49.510307 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-77362071-809d-4b48-8bc5-3fb157ae7e1a None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:49.511902 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-77362071-809d-4b48-8bc5-3fb157ae7e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:49.610782 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 716/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 19:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:49.649682 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceefc7ae-069f-4cfe-becd-df607e8c9cb7 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:49.651230 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-ceefc7ae-069f-4cfe-becd-df607e8c9cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:49.758701 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceefc7ae-069f-4cfe-becd-df607e8c9cb7 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:49.799206 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:49.836831 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c59086-230b-454c-a0ce-ddf6d08bb066 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:49.837445 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c59086-230b-454c-a0ce-ddf6d08bb066 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:49.837727 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c59086-230b-454c-a0ce-ddf6d08bb066 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:49.838003 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c59086-230b-454c-a0ce-ddf6d08bb066 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:49.840272 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e5c59086-230b-454c-a0ce-ddf6d08bb066 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:49.948119 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c59086-230b-454c-a0ce-ddf6d08bb066 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:49.950177 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c59086-230b-454c-a0ce-ddf6d08bb066 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:49.968871 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 733/1449] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:56:52.125099 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72350584-e1eb-4dac-9ac9-2587563f1058 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:52.125667 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72350584-e1eb-4dac-9ac9-2587563f1058 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:52.125667 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72350584-e1eb-4dac-9ac9-2587563f1058 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:52.127010 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 717/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 19:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:56:52.132391 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd12a0b-0350-4637-93ba-5ddeb9c03aab None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:52.132810 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd12a0b-0350-4637-93ba-5ddeb9c03aab None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:52.132989 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd12a0b-0350-4637-93ba-5ddeb9c03aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:52.201589 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-8cd12a0b-0350-4637-93ba-5ddeb9c03aab None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:52.202807 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-8cd12a0b-0350-4637-93ba-5ddeb9c03aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:52.223988 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:52.302351 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 734/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 19:56:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:52.343688 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72350584-e1eb-4dac-9ac9-2587563f1058 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:52.345263 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-72350584-e1eb-4dac-9ac9-2587563f1058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:52.449742 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72350584-e1eb-4dac-9ac9-2587563f1058 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:52.471095 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:52.524571 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bea182-3a4b-48cc-9337-a506e8fbdd6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:52.525189 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bea182-3a4b-48cc-9337-a506e8fbdd6d service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:52.525465 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bea182-3a4b-48cc-9337-a506e8fbdd6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:52.525734 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bea182-3a4b-48cc-9337-a506e8fbdd6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:52.527408 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-a1bea182-3a4b-48cc-9337-a506e8fbdd6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:52.634619 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bea182-3a4b-48cc-9337-a506e8fbdd6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:52.636048 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bea182-3a4b-48cc-9337-a506e8fbdd6d service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:52.655009 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 718/1452] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:56:54.770714 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da241c1-d206-4d70-9206-42c39d1a5bec None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:54.771254 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da241c1-d206-4d70-9206-42c39d1a5bec None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:54.771254 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da241c1-d206-4d70-9206-42c39d1a5bec None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:54.772917 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 735/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 19:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:56:54.778487 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8db50e-1f52-4b7f-be70-edd2199ed1fe None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:54.778904 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8db50e-1f52-4b7f-be70-edd2199ed1fe None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:54.779040 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8db50e-1f52-4b7f-be70-edd2199ed1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:54.844122 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:54.847020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-ad8db50e-1f52-4b7f-be70-edd2199ed1fe None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:54.848951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-ad8db50e-1f52-4b7f-be70-edd2199ed1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:54.948300 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 719/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 19:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:54.986839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da241c1-d206-4d70-9206-42c39d1a5bec None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:54.988436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-2da241c1-d206-4d70-9206-42c39d1a5bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:55.094948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da241c1-d206-4d70-9206-42c39d1a5bec None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:55.146100 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:55.172747 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c25686-d30b-4949-b34a-79d21cc8e372 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:55.173292 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c25686-d30b-4949-b34a-79d21cc8e372 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:55.173571 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c25686-d30b-4949-b34a-79d21cc8e372 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:55.173844 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c25686-d30b-4949-b34a-79d21cc8e372 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:55.175532 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-c3c25686-d30b-4949-b34a-79d21cc8e372 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:55.287031 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c25686-d30b-4949-b34a-79d21cc8e372 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:55.288453 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c25686-d30b-4949-b34a-79d21cc8e372 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:55.307813 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 736/1455] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:56:57.400283 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7415a-4b7e-414a-b455-988adfc7db42 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:57.400835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7415a-4b7e-414a-b455-988adfc7db42 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:57.400835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7415a-4b7e-414a-b455-988adfc7db42 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:57.402834 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 720/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 19:56:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:56:57.408233 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ccf03-6729-472e-ab82-4f90e1961828 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:57.408721 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ccf03-6729-472e-ab82-4f90e1961828 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:57.408829 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ccf03-6729-472e-ab82-4f90e1961828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:57.478517 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-f05ccf03-6729-472e-ab82-4f90e1961828 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:57.479724 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-f05ccf03-6729-472e-ab82-4f90e1961828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:57.508792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:57.577741 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 737/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 19:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:57.616171 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7415a-4b7e-414a-b455-988adfc7db42 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:57.617711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-10b7415a-4b7e-414a-b455-988adfc7db42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:57.727639 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7415a-4b7e-414a-b455-988adfc7db42 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:57.756053 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:57.802803 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f008f5-d3f5-4f67-b363-6c9b64c1abb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:57.803410 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f008f5-d3f5-4f67-b363-6c9b64c1abb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:57.803679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f008f5-d3f5-4f67-b363-6c9b64c1abb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:57.803947 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f008f5-d3f5-4f67-b363-6c9b64c1abb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:57.805614 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-74f008f5-d3f5-4f67-b363-6c9b64c1abb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:57.915016 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f008f5-d3f5-4f67-b363-6c9b64c1abb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:57.916468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f008f5-d3f5-4f67-b363-6c9b64c1abb0 service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:57.935810 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 721/1458] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:57:00.004020 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487dcf4b-d35c-489f-a448-0c111bf051c1 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:00.004637 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487dcf4b-d35c-489f-a448-0c111bf051c1 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:00.004637 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487dcf4b-d35c-489f-a448-0c111bf051c1 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:00.005947 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 738/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 19:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:57:00.012855 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d5d3c-11b2-4362-8aa0-3267f6ba16eb None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:00.013289 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d5d3c-11b2-4362-8aa0-3267f6ba16eb None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:00.013399 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d5d3c-11b2-4362-8aa0-3267f6ba16eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:00.080776 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-6e2d5d3c-11b2-4362-8aa0-3267f6ba16eb None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:00.082131 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-6e2d5d3c-11b2-4362-8aa0-3267f6ba16eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:00.091059 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:00.182610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 722/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 19:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:00.220154 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487dcf4b-d35c-489f-a448-0c111bf051c1 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:00.221708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-487dcf4b-d35c-489f-a448-0c111bf051c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:00.330050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487dcf4b-d35c-489f-a448-0c111bf051c1 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:00.388862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:00.408199 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16854252-f39a-460d-9e14-354059f0faa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:00.408836 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16854252-f39a-460d-9e14-354059f0faa7 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:00.409114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16854252-f39a-460d-9e14-354059f0faa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:00.409384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16854252-f39a-460d-9e14-354059f0faa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:00.411004 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-16854252-f39a-460d-9e14-354059f0faa7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:00.648863 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16854252-f39a-460d-9e14-354059f0faa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:00.650279 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16854252-f39a-460d-9e14-354059f0faa7 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:00.669685 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 739/1461] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:57:02.748924 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd9a80-8d97-41b1-a9ee-196a40ab737b None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:02.749487 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd9a80-8d97-41b1-a9ee-196a40ab737b None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:02.749487 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd9a80-8d97-41b1-a9ee-196a40ab737b None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:02.750902 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 723/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 19:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:57:02.756260 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7637f-b786-4847-954d-3b62b33d1905 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:02.756705 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7637f-b786-4847-954d-3b62b33d1905 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:02.756847 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7637f-b786-4847-954d-3b62b33d1905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:02.822241 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-eac7637f-b786-4847-954d-3b62b33d1905 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:02.823541 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-eac7637f-b786-4847-954d-3b62b33d1905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:02.862966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:02.924761 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 740/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 19:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:02.963973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd9a80-8d97-41b1-a9ee-196a40ab737b None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:02.965604 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-49dd9a80-8d97-41b1-a9ee-196a40ab737b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:03.071589 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd9a80-8d97-41b1-a9ee-196a40ab737b None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:03.117727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:03.157968 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99aa5d6-6b5c-4e98-97a1-a828805f3c2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:03.158581 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99aa5d6-6b5c-4e98-97a1-a828805f3c2e service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:03.158889 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99aa5d6-6b5c-4e98-97a1-a828805f3c2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:03.159163 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99aa5d6-6b5c-4e98-97a1-a828805f3c2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:03.160843 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-a99aa5d6-6b5c-4e98-97a1-a828805f3c2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:03.274718 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99aa5d6-6b5c-4e98-97a1-a828805f3c2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:03.276184 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99aa5d6-6b5c-4e98-97a1-a828805f3c2e service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:03.296823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 724/1464] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:57:05.386339 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a656e2-9562-4934-be6c-bd3422578cfd None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:05.386887 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a656e2-9562-4934-be6c-bd3422578cfd None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:05.386887 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a656e2-9562-4934-be6c-bd3422578cfd None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:05.388235 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 741/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 19:57:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:57:05.393697 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef03bbc-8ec2-4af1-a068-001121b68735 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:05.394182 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef03bbc-8ec2-4af1-a068-001121b68735 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:05.394290 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef03bbc-8ec2-4af1-a068-001121b68735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:05.461279 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-5ef03bbc-8ec2-4af1-a068-001121b68735 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:05.462402 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-5ef03bbc-8ec2-4af1-a068-001121b68735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:05.481657 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:05.561843 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 725/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 19:57:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:05.601694 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a656e2-9562-4934-be6c-bd3422578cfd None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:05.603226 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-53a656e2-9562-4934-be6c-bd3422578cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:05.710342 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a656e2-9562-4934-be6c-bd3422578cfd None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:05.783167 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:05.790335 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882ef91-63d7-4a07-9d7e-825f32fda1e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:05.790948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882ef91-63d7-4a07-9d7e-825f32fda1e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:05.791231 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882ef91-63d7-4a07-9d7e-825f32fda1e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:05.791503 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882ef91-63d7-4a07-9d7e-825f32fda1e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:05.793162 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5882ef91-63d7-4a07-9d7e-825f32fda1e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:05.904048 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5882ef91-63d7-4a07-9d7e-825f32fda1e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:05.905510 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5882ef91-63d7-4a07-9d7e-825f32fda1e7 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:05.924275 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 742/1467] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:57:08.003276 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec4db1-f29f-493b-a5ed-e25178d52360 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:08.003856 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec4db1-f29f-493b-a5ed-e25178d52360 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:08.003856 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec4db1-f29f-493b-a5ed-e25178d52360 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:08.005217 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 726/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 19:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:57:08.010353 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecacc0e9-25a7-4ccc-b16c-362666849938 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:08.010755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecacc0e9-25a7-4ccc-b16c-362666849938 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:08.010933 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecacc0e9-25a7-4ccc-b16c-362666849938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:08.076548 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-ecacc0e9-25a7-4ccc-b16c-362666849938 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:08.077839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-ecacc0e9-25a7-4ccc-b16c-362666849938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:08.127533 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:08.176294 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 743/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 19:57:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:08.214533 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec4db1-f29f-493b-a5ed-e25178d52360 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:08.216073 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-30ec4db1-f29f-493b-a5ed-e25178d52360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:08.318744 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec4db1-f29f-493b-a5ed-e25178d52360 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:08.366755 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:08.392812 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3943b02e-5136-47e8-9a0c-1b4df937858e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:08.393423 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943b02e-5136-47e8-9a0c-1b4df937858e service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:08.393703 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943b02e-5136-47e8-9a0c-1b4df937858e service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:08.393972 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943b02e-5136-47e8-9a0c-1b4df937858e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:08.395583 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-3943b02e-5136-47e8-9a0c-1b4df937858e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:08.506610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3943b02e-5136-47e8-9a0c-1b4df937858e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:08.508030 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3943b02e-5136-47e8-9a0c-1b4df937858e service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:08.527102 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 727/1470] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:57:18.174495 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=22 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:18.524206 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=66 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:28.539436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b52b3-80da-4857-8e57-310799cb5244 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:28.539966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b52b3-80da-4857-8e57-310799cb5244 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:28.539966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b52b3-80da-4857-8e57-310799cb5244 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:28.547568 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 744/1471] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:57:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:57:28.547770 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a846c76e-1456-4981-9986-750590b147b5 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:28.547770 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a846c76e-1456-4981-9986-750590b147b5 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:28.547770 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a846c76e-1456-4981-9986-750590b147b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:28.622016 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-a846c76e-1456-4981-9986-750590b147b5 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:28.623516 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-a846c76e-1456-4981-9986-750590b147b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:28.730031 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 728/1472] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:57:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:28.812324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec359814-3c15-4201-a99d-5d81c4078c75 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:28.812650 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec359814-3c15-4201-a99d-5d81c4078c75 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:28.812933 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec359814-3c15-4201-a99d-5d81c4078c75 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:28.814278 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 745/1473] 10.222.0.39 () {54 vars in 966 bytes} [Fri Aug 5 19:57:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:57:28.819313 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7dcda7-9e1c-47c4-bb1e-21261bd832ba None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:28.819595 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7dcda7-9e1c-47c4-bb1e-21261bd832ba None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:28.819871 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7dcda7-9e1c-47c4-bb1e-21261bd832ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:28.901153 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-cc7dcda7-9e1c-47c4-bb1e-21261bd832ba None None] MFA Rules not processed for user `2fe69b398c1f4e18b625a63ae40b01e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:28.902394 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-cc7dcda7-9e1c-47c4-bb1e-21261bd832ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:28.923377 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:28.981316 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 729/1474] 10.222.0.39 () {58 vars in 1063 bytes} [Fri Aug 5 19:57:28 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:28.988927 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec359814-3c15-4201-a99d-5d81c4078c75 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:28.990440 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-ec359814-3c15-4201-a99d-5d81c4078c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:29.094540 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec359814-3c15-4201-a99d-5d81c4078c75 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:29.171532 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ade8b5-ce18-4dc1-b079-b621439d1819 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe69b398c1f4e18b625a63ae40b01e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:29.172156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ade8b5-ce18-4dc1-b079-b621439d1819 service glance] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:29.172441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ade8b5-ce18-4dc1-b079-b621439d1819 service glance] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:29.172760 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ade8b5-ce18-4dc1-b079-b621439d1819 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:29.174490 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-36ade8b5-ce18-4dc1-b079-b621439d1819 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:29.232662 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:29.291034 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ade8b5-ce18-4dc1-b079-b621439d1819 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:29.293049 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ade8b5-ce18-4dc1-b079-b621439d1819 service glance] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:29.312836 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 746/1475] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:57:31.427549 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18614b0a-8591-42b6-8dfe-8f6b2aca6873 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:31.428061 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18614b0a-8591-42b6-8dfe-8f6b2aca6873 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:31.428061 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18614b0a-8591-42b6-8dfe-8f6b2aca6873 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:31.429390 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 730/1476] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:57:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:57:31.434992 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e94064-ca64-4ecb-af10-213527f28243 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:31.435427 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e94064-ca64-4ecb-af10-213527f28243 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:31.435592 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e94064-ca64-4ecb-af10-213527f28243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:31.502150 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-76e94064-ca64-4ecb-af10-213527f28243 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:31.503369 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-76e94064-ca64-4ecb-af10-213527f28243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:31.601370 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 747/1477] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:57:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:31.624866 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18614b0a-8591-42b6-8dfe-8f6b2aca6873 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:31.626418 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-18614b0a-8591-42b6-8dfe-8f6b2aca6873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:31.773860 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18614b0a-8591-42b6-8dfe-8f6b2aca6873 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:31.826103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:31.878817 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b9d31-a51c-494b-a3d2-d9f0c32b512d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:31.879420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b9d31-a51c-494b-a3d2-d9f0c32b512d admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:31.879687 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b9d31-a51c-494b-a3d2-d9f0c32b512d admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:31.879961 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b9d31-a51c-494b-a3d2-d9f0c32b512d admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:31.880686 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b9d31-a51c-494b-a3d2-d9f0c32b512d admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:31.882435 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: WARNING py.warnings [None req-5b0b9d31-a51c-494b-a3d2-d9f0c32b512d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 19:57:31.882435 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: warnings.warn(msg) Aug 05 19:57:31.882435 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: Aug 05 19:57:31.883555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b9d31-a51c-494b-a3d2-d9f0c32b512d admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:31.904765 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 731/1478] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Aug 5 19:57:31 2022] GET /identity/v3/projects => generated 2553 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:57:33.979901 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec846b-a192-4b8b-9dc4-c015052c843d None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:33.980703 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec846b-a192-4b8b-9dc4-c015052c843d None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:33.980817 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec846b-a192-4b8b-9dc4-c015052c843d None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:33.982501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 748/1479] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:57:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:57:33.988282 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3291d5e-ad82-4904-baf5-4e3180de9d31 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:33.988766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3291d5e-ad82-4904-baf5-4e3180de9d31 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:33.988873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3291d5e-ad82-4904-baf5-4e3180de9d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:34.057797 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-a3291d5e-ad82-4904-baf5-4e3180de9d31 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:34.058997 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-a3291d5e-ad82-4904-baf5-4e3180de9d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:34.171073 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 732/1480] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:57:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:34.216025 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec846b-a192-4b8b-9dc4-c015052c843d None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:34.217620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-58ec846b-a192-4b8b-9dc4-c015052c843d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:34.227518 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:34.367541 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec846b-a192-4b8b-9dc4-c015052c843d None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:34.459321 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:34.493190 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f6e25-4ba8-4126-a01f-263da5231419 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:34.493809 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f6e25-4ba8-4126-a01f-263da5231419 service nova] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:34.494103 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f6e25-4ba8-4126-a01f-263da5231419 service nova] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:34.494372 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f6e25-4ba8-4126-a01f-263da5231419 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:34.497027 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-735f6e25-4ba8-4126-a01f-263da5231419 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:34.588608 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f6e25-4ba8-4126-a01f-263da5231419 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:34.590220 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f6e25-4ba8-4126-a01f-263da5231419 service nova] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:34.609031 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 749/1481] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:57:36.983590 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d7f8e-7777-4261-8243-3816f3277c21 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:36.984197 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d7f8e-7777-4261-8243-3816f3277c21 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:36.984197 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d7f8e-7777-4261-8243-3816f3277c21 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:36.985582 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 733/1482] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:57:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:57:36.991122 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bafb0-f905-4d31-82b1-102ff49084d4 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:36.991564 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bafb0-f905-4d31-82b1-102ff49084d4 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:36.991672 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bafb0-f905-4d31-82b1-102ff49084d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:37.060327 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-a50bafb0-f905-4d31-82b1-102ff49084d4 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:37.061695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-a50bafb0-f905-4d31-82b1-102ff49084d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:37.108651 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:37.160911 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 750/1483] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:57:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:37.200599 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d7f8e-7777-4261-8243-3816f3277c21 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:37.202277 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-365d7f8e-7777-4261-8243-3816f3277c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:37.226243 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:37.311375 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d7f8e-7777-4261-8243-3816f3277c21 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:37.389105 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250510b4-997a-49d0-a657-2af445aa9034 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:37.389719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250510b4-997a-49d0-a657-2af445aa9034 service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:37.390009 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250510b4-997a-49d0-a657-2af445aa9034 service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:37.390264 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250510b4-997a-49d0-a657-2af445aa9034 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:37.391886 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-250510b4-997a-49d0-a657-2af445aa9034 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:37.471005 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:37.507084 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250510b4-997a-49d0-a657-2af445aa9034 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:37.508504 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250510b4-997a-49d0-a657-2af445aa9034 service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:37.527565 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 734/1484] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:57:39.661530 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c254d52-cfe9-4e1d-8dc5-343da1375c67 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:39.662090 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c254d52-cfe9-4e1d-8dc5-343da1375c67 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:39.662090 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c254d52-cfe9-4e1d-8dc5-343da1375c67 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:39.663421 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 751/1485] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:57:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:57:39.669997 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b9629-1493-468a-a3cc-f38100aab26a None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:39.670448 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b9629-1493-468a-a3cc-f38100aab26a None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:39.670448 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b9629-1493-468a-a3cc-f38100aab26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:39.737826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-2d1b9629-1493-468a-a3cc-f38100aab26a None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:39.739181 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-2d1b9629-1493-468a-a3cc-f38100aab26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:39.839632 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 735/1486] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:39.878264 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c254d52-cfe9-4e1d-8dc5-343da1375c67 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:39.879796 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-7c254d52-cfe9-4e1d-8dc5-343da1375c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:39.988216 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c254d52-cfe9-4e1d-8dc5-343da1375c67 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:40.065226 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:40.067200 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861cc7d-8d27-4411-bca5-3378c3faba80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:40.067872 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861cc7d-8d27-4411-bca5-3378c3faba80 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:40.068360 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861cc7d-8d27-4411-bca5-3378c3faba80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:40.068700 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861cc7d-8d27-4411-bca5-3378c3faba80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:40.071357 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-d861cc7d-8d27-4411-bca5-3378c3faba80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:40.183848 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d861cc7d-8d27-4411-bca5-3378c3faba80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:40.185337 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d861cc7d-8d27-4411-bca5-3378c3faba80 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:40.204787 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 752/1487] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:57:42.428408 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e432c-7a67-4102-af61-229e930e76b5 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:42.428980 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e432c-7a67-4102-af61-229e930e76b5 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:42.428980 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e432c-7a67-4102-af61-229e930e76b5 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:42.430280 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 736/1488] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:57:42.436317 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf3d88-78b0-43d8-bcab-4bbd9502bd69 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:42.436787 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf3d88-78b0-43d8-bcab-4bbd9502bd69 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:42.436904 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf3d88-78b0-43d8-bcab-4bbd9502bd69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:42.505489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-e2bf3d88-78b0-43d8-bcab-4bbd9502bd69 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:42.506697 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e2bf3d88-78b0-43d8-bcab-4bbd9502bd69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:42.561890 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:42.607327 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 753/1489] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:57:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:42.647553 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e432c-7a67-4102-af61-229e930e76b5 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:42.649437 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-9e8e432c-7a67-4102-af61-229e930e76b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:42.687046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:42.757213 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e432c-7a67-4102-af61-229e930e76b5 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:42.831536 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195337b-5a91-4b5f-80e1-1521a4d4b414 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:42.832144 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195337b-5a91-4b5f-80e1-1521a4d4b414 service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:42.832419 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195337b-5a91-4b5f-80e1-1521a4d4b414 service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:42.832719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195337b-5a91-4b5f-80e1-1521a4d4b414 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:42.834337 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-b195337b-5a91-4b5f-80e1-1521a4d4b414 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:42.917745 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:42.947534 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195337b-5a91-4b5f-80e1-1521a4d4b414 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:42.949640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195337b-5a91-4b5f-80e1-1521a4d4b414 service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:42.967843 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 737/1490] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:57:45.253132 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27bbca-50e1-45ca-9df5-994a096e89bd None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:45.253735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27bbca-50e1-45ca-9df5-994a096e89bd None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:45.253867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27bbca-50e1-45ca-9df5-994a096e89bd None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:45.255196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 754/1491] 10.222.0.39 () {54 vars in 927 bytes} [Fri Aug 5 19:57:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:57:45.261058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d2d8f-c533-44c1-b3cf-f340f2872e58 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:45.261583 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d2d8f-c533-44c1-b3cf-f340f2872e58 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:45.261583 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d2d8f-c533-44c1-b3cf-f340f2872e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:45.330308 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-c48d2d8f-c533-44c1-b3cf-f340f2872e58 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:45.332660 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-c48d2d8f-c533-44c1-b3cf-f340f2872e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:45.432524 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 738/1492] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Aug 5 19:57:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:57:45.472845 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da27bbca-50e1-45ca-9df5-994a096e89bd None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:45.474406 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-da27bbca-50e1-45ca-9df5-994a096e89bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:45.579269 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da27bbca-50e1-45ca-9df5-994a096e89bd None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:45.653945 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab2b5b-5183-40c2-8d78-0774bf83a22f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:45.654554 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab2b5b-5183-40c2-8d78-0774bf83a22f service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:45.654857 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab2b5b-5183-40c2-8d78-0774bf83a22f service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:45.655108 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab2b5b-5183-40c2-8d78-0774bf83a22f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:45.656782 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-c0ab2b5b-5183-40c2-8d78-0774bf83a22f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:45.666587 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:45.776186 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab2b5b-5183-40c2-8d78-0774bf83a22f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:45.777781 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab2b5b-5183-40c2-8d78-0774bf83a22f service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:45.796080 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 755/1493] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:57:55.430255 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=82 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:55.793933 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=50 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:58.188703 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5a48a-4b0f-4e0a-8ad6-ae9b75b63592 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:58.189300 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5a48a-4b0f-4e0a-8ad6-ae9b75b63592 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:58.189300 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5a48a-4b0f-4e0a-8ad6-ae9b75b63592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:58.265389 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-b8b5a48a-4b0f-4e0a-8ad6-ae9b75b63592 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:58.266603 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-b8b5a48a-4b0f-4e0a-8ad6-ae9b75b63592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:58.364424 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 739/1494] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 19:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:58.372814 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcde763-85ef-464a-ab2b-c519309d0b08 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:58.373341 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcde763-85ef-464a-ab2b-c519309d0b08 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:58.373437 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcde763-85ef-464a-ab2b-c519309d0b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:58.453449 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-4dcde763-85ef-464a-ab2b-c519309d0b08 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:58.454571 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-4dcde763-85ef-464a-ab2b-c519309d0b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:58.550683 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 756/1495] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 19:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:58.555476 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b5a48a-4b0f-4e0a-8ad6-ae9b75b63592 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:58.556715 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-b8b5a48a-4b0f-4e0a-8ad6-ae9b75b63592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:58.567982 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b5a48a-4b0f-4e0a-8ad6-ae9b75b63592 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:58.631714 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:58.668196 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea5a495-1379-4292-834a-a2ca4da224bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:58.668851 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea5a495-1379-4292-834a-a2ca4da224bd admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:58.669132 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea5a495-1379-4292-834a-a2ca4da224bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:58.669407 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea5a495-1379-4292-834a-a2ca4da224bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:58.670101 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea5a495-1379-4292-834a-a2ca4da224bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:58:58.670407 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea5a495-1379-4292-834a-a2ca4da224bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:58.671440 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: WARNING py.warnings [None req-8ea5a495-1379-4292-834a-a2ca4da224bd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 05 19:58:58.671440 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: warnings.warn(msg) Aug 05 19:58:58.671440 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: Aug 05 19:58:58.672329 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea5a495-1379-4292-834a-a2ca4da224bd admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:58.683697 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 740/1496] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 19:58:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:58:58.692194 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcde763-85ef-464a-ab2b-c519309d0b08 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:58.702181 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcde763-85ef-464a-ab2b-c519309d0b08 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:58.764135 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:58.798188 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa800be7-31f5-46bb-a584-b95d2f129f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:58.798804 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa800be7-31f5-46bb-a584-b95d2f129f61 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:58.799084 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa800be7-31f5-46bb-a584-b95d2f129f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:58.799384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa800be7-31f5-46bb-a584-b95d2f129f61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:58.800051 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa800be7-31f5-46bb-a584-b95d2f129f61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:58.801834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: WARNING py.warnings [None req-fa800be7-31f5-46bb-a584-b95d2f129f61 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 19:58:58.801834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: warnings.warn(msg) Aug 05 19:58:58.801834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: Aug 05 19:58:58.803029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa800be7-31f5-46bb-a584-b95d2f129f61 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:58.859558 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 757/1497] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 19:58:58 2022] POST /identity/v3/projects => generated 377 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:58.863959 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea5a495-1379-4292-834a-a2ca4da224bd admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:58.865498 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-8ea5a495-1379-4292-834a-a2ca4da224bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:58.899799 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea5a495-1379-4292-834a-a2ca4da224bd admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:58.995991 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694415bf-0c97-49c4-8c0f-c8d4eee25d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:58.996623 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694415bf-0c97-49c4-8c0f-c8d4eee25d03 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:58.996899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694415bf-0c97-49c4-8c0f-c8d4eee25d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:58.997170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694415bf-0c97-49c4-8c0f-c8d4eee25d03 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:58.997814 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694415bf-0c97-49c4-8c0f-c8d4eee25d03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:58.998808 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: WARNING py.warnings [None req-694415bf-0c97-49c4-8c0f-c8d4eee25d03 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 19:58:58.998808 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: warnings.warn(msg) Aug 05 19:58:58.998808 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: Aug 05 19:58:59.000012 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694415bf-0c97-49c4-8c0f-c8d4eee25d03 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:59.042697 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 741/1498] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 19:58:58 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:59.046759 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa800be7-31f5-46bb-a584-b95d2f129f61 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:59.048261 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-fa800be7-31f5-46bb-a584-b95d2f129f61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:59.115144 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa800be7-31f5-46bb-a584-b95d2f129f61 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:59.198383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:59.209356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b18b7-bb89-4e54-abb8-d581394614eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:59.209969 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b18b7-bb89-4e54-abb8-d581394614eb admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:59.210243 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b18b7-bb89-4e54-abb8-d581394614eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:59.210508 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b18b7-bb89-4e54-abb8-d581394614eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:59.211442 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928b18b7-bb89-4e54-abb8-d581394614eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:59.212434 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: WARNING py.warnings [None req-928b18b7-bb89-4e54-abb8-d581394614eb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 05 19:58:59.212434 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: warnings.warn(msg) Aug 05 19:58:59.212434 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: Aug 05 19:58:59.213336 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928b18b7-bb89-4e54-abb8-d581394614eb admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:59.225658 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 758/1499] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:58:59.230454 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694415bf-0c97-49c4-8c0f-c8d4eee25d03 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:59.231983 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-694415bf-0c97-49c4-8c0f-c8d4eee25d03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:59.244406 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694415bf-0c97-49c4-8c0f-c8d4eee25d03 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:59.270224 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:59.342838 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec59ab-2deb-4636-a0d8-3ea4359ec84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:59.343359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec59ab-2deb-4636-a0d8-3ea4359ec84e admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:59.343638 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec59ab-2deb-4636-a0d8-3ea4359ec84e admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:59.343904 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec59ab-2deb-4636-a0d8-3ea4359ec84e admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:59.344602 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec59ab-2deb-4636-a0d8-3ea4359ec84e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:59.345620 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: WARNING py.warnings [None req-8cec59ab-2deb-4636-a0d8-3ea4359ec84e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 05 19:58:59.345620 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: warnings.warn(msg) Aug 05 19:58:59.345620 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: Aug 05 19:58:59.346498 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec59ab-2deb-4636-a0d8-3ea4359ec84e admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:59.359594 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 742/1500] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:58:59.363931 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b18b7-bb89-4e54-abb8-d581394614eb admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:59.365520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-928b18b7-bb89-4e54-abb8-d581394614eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:59.400115 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b18b7-bb89-4e54-abb8-d581394614eb admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:59.493877 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7041efa-9f8d-400f-8725-4f61061f83ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:59.494535 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7041efa-9f8d-400f-8725-4f61061f83ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:59.494809 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7041efa-9f8d-400f-8725-4f61061f83ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:59.495078 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7041efa-9f8d-400f-8725-4f61061f83ee admin admin] PATH_INFO: `/v3/projects/e628541a0a314834b272cb71537afe82/users/646634b27a574fffbd81322f06776d1e/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:59.518004 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7041efa-9f8d-400f-8725-4f61061f83ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=e628541a0a314834b272cb71537afe82, user_id=646634b27a574fffbd81322f06776d1e, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:59.519034 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: WARNING py.warnings [None req-c7041efa-9f8d-400f-8725-4f61061f83ee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 19:58:59.519034 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: warnings.warn(msg) Aug 05 19:58:59.519034 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: Aug 05 19:58:59.521092 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7041efa-9f8d-400f-8725-4f61061f83ee admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:59.548651 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 759/1501] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 19:58:59 2022] PUT /identity/v3/projects/e628541a0a314834b272cb71537afe82/users/646634b27a574fffbd81322f06776d1e/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:59.561717 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3687895-c36e-4db2-96e1-12596e3e3f2f None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:59.562010 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3687895-c36e-4db2-96e1-12596e3e3f2f None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:59.562292 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3687895-c36e-4db2-96e1-12596e3e3f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:59.621861 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-b3687895-c36e-4db2-96e1-12596e3e3f2f None None] MFA Rules not processed for user `646634b27a574fffbd81322f06776d1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:59.622994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-b3687895-c36e-4db2-96e1-12596e3e3f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:59.702481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 743/1502] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 19:58:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:59.716409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4e3ab-79bb-431c-af8d-5144a84e4a7e None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:59.716727 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4e3ab-79bb-431c-af8d-5144a84e4a7e None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:59.717001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4e3ab-79bb-431c-af8d-5144a84e4a7e None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:59.718368 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 760/1503] 10.222.0.39 () {54 vars in 964 bytes} [Fri Aug 5 19:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:58:59.723331 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645c47e-24a9-48ee-89ac-b024ebde59ee None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:59.723616 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645c47e-24a9-48ee-89ac-b024ebde59ee None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:59.723895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645c47e-24a9-48ee-89ac-b024ebde59ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:59.755315 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:59.801484 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-f645c47e-24a9-48ee-89ac-b024ebde59ee None None] MFA Rules not processed for user `44dfe45766d743558ed82e464721cf72`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:59.802714 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-f645c47e-24a9-48ee-89ac-b024ebde59ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:59.878488 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 744/1504] 10.222.0.39 () {58 vars in 1061 bytes} [Fri Aug 5 19:58:59 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:59.887003 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca4e3ab-79bb-431c-af8d-5144a84e4a7e None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:59.888513 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-4ca4e3ab-79bb-431c-af8d-5144a84e4a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:59.938835 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:59.993786 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca4e3ab-79bb-431c-af8d-5144a84e4a7e None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:59:00.067078 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ac125-3d2b-46e0-9dc8-8dc4269fffaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:59:00.067684 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ac125-3d2b-46e0-9dc8-8dc4269fffaa service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:59:00.067958 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ac125-3d2b-46e0-9dc8-8dc4269fffaa service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:59:00.068226 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ac125-3d2b-46e0-9dc8-8dc4269fffaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:59:00.069856 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-a43ac125-3d2b-46e0-9dc8-8dc4269fffaa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:59:00.170102 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ac125-3d2b-46e0-9dc8-8dc4269fffaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:59:00.171522 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ac125-3d2b-46e0-9dc8-8dc4269fffaa service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:59:00.174988 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:59:00.191129 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 761/1505] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:58:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:59:00.237487 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253fc7e-b5c8-4f5c-8a5e-10b612b65696 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:59:00.237937 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253fc7e-b5c8-4f5c-8a5e-10b612b65696 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:59:00.238037 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253fc7e-b5c8-4f5c-8a5e-10b612b65696 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:59:00.239367 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 745/1506] 10.222.0.39 () {52 vars in 830 bytes} [Fri Aug 5 19:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:59:00.243552 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ac125-3d2b-46e0-9dc8-8dc4269fffaa service cinder] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:59:00.244988 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-a43ac125-3d2b-46e0-9dc8-8dc4269fffaa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:59:00.344929 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ac125-3d2b-46e0-9dc8-8dc4269fffaa service cinder] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:59:00.438825 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b8c0f-9bf5-4114-a4d3-1c4e0cd8e4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:59:00.439499 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b8c0f-9bf5-4114-a4d3-1c4e0cd8e4d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:59:00.439778 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b8c0f-9bf5-4114-a4d3-1c4e0cd8e4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:59:00.440257 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b8c0f-9bf5-4114-a4d3-1c4e0cd8e4d5 admin admin] PATH_INFO: `/v3/users/646634b27a574fffbd81322f06776d1e` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:59:00.453974 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b8c0f-9bf5-4114-a4d3-1c4e0cd8e4d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=646634b27a574fffbd81322f06776d1e)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:59:00.454999 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: WARNING py.warnings [None req-db9b8c0f-9bf5-4114-a4d3-1c4e0cd8e4d5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 19:59:00.454999 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: warnings.warn(msg) Aug 05 19:59:00.454999 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: Aug 05 19:59:00.456391 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b8c0f-9bf5-4114-a4d3-1c4e0cd8e4d5 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:59:00.466829 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:59:00.525943 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-db9b8c0f-9bf5-4114-a4d3-1c4e0cd8e4d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '646634b27a574fffbd81322f06776d1e'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:59:00.535443 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-db9b8c0f-9bf5-4114-a4d3-1c4e0cd8e4d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '646634b27a574fffbd81322f06776d1e'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:59:00.550261 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-db9b8c0f-9bf5-4114-a4d3-1c4e0cd8e4d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '646634b27a574fffbd81322f06776d1e'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:59:00.573362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-db9b8c0f-9bf5-4114-a4d3-1c4e0cd8e4d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '646634b27a574fffbd81322f06776d1e'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:59:00.573943 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-db9b8c0f-9bf5-4114-a4d3-1c4e0cd8e4d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '646634b27a574fffbd81322f06776d1e'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:59:00.579783 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 762/1507] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 19:59:00 2022] DELETE /identity/v3/users/646634b27a574fffbd81322f06776d1e => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:59:00.584253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8547a-83de-40ff-bf21-29ab9d48aad5 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:59:00.584544 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8547a-83de-40ff-bf21-29ab9d48aad5 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:59:00.584851 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8547a-83de-40ff-bf21-29ab9d48aad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:59:00.645637 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-9cb8547a-83de-40ff-bf21-29ab9d48aad5 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:59:00.646950 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-9cb8547a-83de-40ff-bf21-29ab9d48aad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:59:00.722885 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:59:00.736415 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 746/1508] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 19:59:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:59:00.746312 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b8c0f-9bf5-4114-a4d3-1c4e0cd8e4d5 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:59:00.747542 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-db9b8c0f-9bf5-4114-a4d3-1c4e0cd8e4d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:59:00.850341 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b8c0f-9bf5-4114-a4d3-1c4e0cd8e4d5 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:59:00.905687 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:59:00.926708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d7d30-cf02-42bd-bb5d-96060c02a15f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:59:00.927339 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d7d30-cf02-42bd-bb5d-96060c02a15f service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:59:00.927613 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d7d30-cf02-42bd-bb5d-96060c02a15f service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:59:00.927884 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d7d30-cf02-42bd-bb5d-96060c02a15f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:59:00.929596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-393d7d30-cf02-42bd-bb5d-96060c02a15f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:59:01.041843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d7d30-cf02-42bd-bb5d-96060c02a15f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:59:01.043204 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d7d30-cf02-42bd-bb5d-96060c02a15f service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:59:01.064751 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 763/1509] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 19:59:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:59:01.400443 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8547a-83de-40ff-bf21-29ab9d48aad5 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:59:01.401782 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-9cb8547a-83de-40ff-bf21-29ab9d48aad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:59:01.413327 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8547a-83de-40ff-bf21-29ab9d48aad5 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:59:01.514453 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f7944e-dec1-4212-ac8d-e05c32ebd67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:59:01.515094 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7944e-dec1-4212-ac8d-e05c32ebd67b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:59:01.515376 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7944e-dec1-4212-ac8d-e05c32ebd67b admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:59:01.515652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7944e-dec1-4212-ac8d-e05c32ebd67b admin admin] PATH_INFO: `/v3/projects/e628541a0a314834b272cb71537afe82` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:59:01.526385 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f7944e-dec1-4212-ac8d-e05c32ebd67b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e628541a0a314834b272cb71537afe82)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:59:01.527407 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: WARNING py.warnings [None req-a8f7944e-dec1-4212-ac8d-e05c32ebd67b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 19:59:01.527407 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: warnings.warn(msg) Aug 05 19:59:01.527407 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: Aug 05 19:59:01.528778 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f7944e-dec1-4212-ac8d-e05c32ebd67b admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:59:01.587471 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-a8f7944e-dec1-4212-ac8d-e05c32ebd67b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e628541a0a314834b272cb71537afe82', 'request_id': 'req-a8f7944e-dec1-4212-ac8d-e05c32ebd67b'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:59:01.596628 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-a8f7944e-dec1-4212-ac8d-e05c32ebd67b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e628541a0a314834b272cb71537afe82', 'request_id': 'req-a8f7944e-dec1-4212-ac8d-e05c32ebd67b'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:59:01.604224 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-a8f7944e-dec1-4212-ac8d-e05c32ebd67b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e628541a0a314834b272cb71537afe82', 'request_id': 'req-a8f7944e-dec1-4212-ac8d-e05c32ebd67b'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:59:01.636380 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-a8f7944e-dec1-4212-ac8d-e05c32ebd67b admin admin] The token cache is being invalidate because project e628541a0a314834b272cb71537afe82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:59:01.636690 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-a8f7944e-dec1-4212-ac8d-e05c32ebd67b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:59:01.637122 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-a8f7944e-dec1-4212-ac8d-e05c32ebd67b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:59:01.638546 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 747/1510] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 19:59:01 2022] DELETE /identity/v3/projects/e628541a0a314834b272cb71537afe82 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:59:11.062472 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=62 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:59:11.634346 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:00:34.918111 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b1dc8-f932-4ddb-97a3-c158e2436aed None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:34.918714 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b1dc8-f932-4ddb-97a3-c158e2436aed None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:34.918714 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b1dc8-f932-4ddb-97a3-c158e2436aed None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:34.920064 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 764/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 20:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 20:00:34.925162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a39819-4210-4759-89b3-226f4cfdf977 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:34.925564 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a39819-4210-4759-89b3-226f4cfdf977 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:34.925719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a39819-4210-4759-89b3-226f4cfdf977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:35.010916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-d1a39819-4210-4759-89b3-226f4cfdf977 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:00:35.012128 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-d1a39819-4210-4759-89b3-226f4cfdf977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:00:35.118689 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 748/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 20:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:00:35.284791 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b1dc8-f932-4ddb-97a3-c158e2436aed None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:00:35.286872 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-7d2b1dc8-f932-4ddb-97a3-c158e2436aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:00:35.412355 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b1dc8-f932-4ddb-97a3-c158e2436aed None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:00:35.418535 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f74d83-9930-4e78-bf5d-d48d70d3089a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:00:35.419142 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f74d83-9930-4e78-bf5d-d48d70d3089a service nova] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:35.419420 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f74d83-9930-4e78-bf5d-d48d70d3089a service nova] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:35.419691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f74d83-9930-4e78-bf5d-d48d70d3089a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:35.427695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f74d83-9930-4e78-bf5d-d48d70d3089a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:00:35.429844 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f74d83-9930-4e78-bf5d-d48d70d3089a service nova] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:00:35.438982 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 765/1513] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:00:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:00:36.299002 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7e350-6862-4d67-8f90-260580863874 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:36.299532 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7e350-6862-4d67-8f90-260580863874 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:36.299532 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7e350-6862-4d67-8f90-260580863874 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:36.300845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 749/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 20:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 20:00:36.306534 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7eda90-19f4-4710-80a2-61178bcad9b3 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:36.306987 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7eda90-19f4-4710-80a2-61178bcad9b3 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:36.307097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7eda90-19f4-4710-80a2-61178bcad9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:36.327116 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-da7eda90-19f4-4710-80a2-61178bcad9b3 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:00:36.328307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-da7eda90-19f4-4710-80a2-61178bcad9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:00:36.336390 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 766/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 20:00:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:00:36.494236 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7e350-6862-4d67-8f90-260580863874 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:00:36.509137 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7e350-6862-4d67-8f90-260580863874 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:00:36.514759 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a0f5a-d3d8-42d8-a53e-7be5e46260f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:00:36.515372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a0f5a-d3d8-42d8-a53e-7be5e46260f5 service nova] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:36.515649 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a0f5a-d3d8-42d8-a53e-7be5e46260f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:36.515921 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a0f5a-d3d8-42d8-a53e-7be5e46260f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:36.524556 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a0f5a-d3d8-42d8-a53e-7be5e46260f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:00:36.526151 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a0f5a-d3d8-42d8-a53e-7be5e46260f5 service nova] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:00:36.535471 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 750/1516] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:00:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:00:36.775540 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0899770-6b9b-4d3c-8846-c57ad37857a6 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:36.775850 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0899770-6b9b-4d3c-8846-c57ad37857a6 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:36.776154 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0899770-6b9b-4d3c-8846-c57ad37857a6 None None] PATH_INFO: `/v3` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:36.777532 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 767/1517] 10.222.0.87 () {54 vars in 925 bytes} [Fri Aug 5 20:00:36 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 20:00:36.783742 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d1972-7ef0-4b7a-be22-143966f7057c None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:36.784031 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d1972-7ef0-4b7a-be22-143966f7057c None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:36.784309 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d1972-7ef0-4b7a-be22-143966f7057c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:36.843212 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-789d1972-7ef0-4b7a-be22-143966f7057c None None] MFA Rules not processed for user `c61e2c954f9a49abad9f3de17ee39f63`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:00:36.844315 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-789d1972-7ef0-4b7a-be22-143966f7057c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:00:36.861225 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:00:36.918118 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 751/1518] 10.222.0.87 () {58 vars in 1015 bytes} [Fri Aug 5 20:00:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:00:46.187922 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264dfd73-442d-4e59-bf5b-d41b70e85186 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:46.188476 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264dfd73-442d-4e59-bf5b-d41b70e85186 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:46.188476 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264dfd73-442d-4e59-bf5b-d41b70e85186 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:46.189755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 768/1519] 10.222.0.39 () {54 vars in 961 bytes} [Fri Aug 5 20:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 20:00:46.194959 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87168e38-4942-4278-a181-0b6cfd76fe28 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:46.195405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87168e38-4942-4278-a181-0b6cfd76fe28 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:46.195517 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87168e38-4942-4278-a181-0b6cfd76fe28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:46.216619 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-87168e38-4942-4278-a181-0b6cfd76fe28 None None] MFA Rules not processed for user `c61e2c954f9a49abad9f3de17ee39f63`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:00:46.217908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-87168e38-4942-4278-a181-0b6cfd76fe28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:00:46.227275 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 752/1520] 10.222.0.39 () {58 vars in 1058 bytes} [Fri Aug 5 20:00:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:00:46.234835 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264dfd73-442d-4e59-bf5b-d41b70e85186 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:00:46.248591 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264dfd73-442d-4e59-bf5b-d41b70e85186 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:00:46.253641 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f10c55-3ec3-46de-9081-3c06da897802 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61e2c954f9a49abad9f3de17ee39f63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:00:46.254233 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f10c55-3ec3-46de-9081-3c06da897802 service placement] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:46.254514 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f10c55-3ec3-46de-9081-3c06da897802 service placement] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:46.254789 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f10c55-3ec3-46de-9081-3c06da897802 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:46.264027 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f10c55-3ec3-46de-9081-3c06da897802 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:00:46.265644 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f10c55-3ec3-46de-9081-3c06da897802 service placement] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:00:46.275915 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 769/1521] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:00:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:00:47.769465 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f949a5-5aed-468e-93e1-eda0ff7490d2 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:47.770050 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f949a5-5aed-468e-93e1-eda0ff7490d2 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:47.770050 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f949a5-5aed-468e-93e1-eda0ff7490d2 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:47.771363 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 753/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 20:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 20:00:47.776994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7cccb-313f-422d-aaab-8faebc323734 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:47.777416 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7cccb-313f-422d-aaab-8faebc323734 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:47.777549 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7cccb-313f-422d-aaab-8faebc323734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:47.797506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-e8f7cccb-313f-422d-aaab-8faebc323734 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:00:47.798718 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e8f7cccb-313f-422d-aaab-8faebc323734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:00:47.806995 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 770/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 20:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:00:47.961819 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f949a5-5aed-468e-93e1-eda0ff7490d2 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:00:47.976189 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f949a5-5aed-468e-93e1-eda0ff7490d2 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:00:47.981670 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d6ea6-1748-422a-9fb5-1959380cef50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:00:47.982274 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d6ea6-1748-422a-9fb5-1959380cef50 service nova] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:47.982555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d6ea6-1748-422a-9fb5-1959380cef50 service nova] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:47.982867 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d6ea6-1748-422a-9fb5-1959380cef50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:47.990949 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5d6ea6-1748-422a-9fb5-1959380cef50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:00:47.992695 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5d6ea6-1748-422a-9fb5-1959380cef50 service nova] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:00:48.002234 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 754/1524] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:00:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:00:48.843730 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd5c6c-1dd2-467e-9eee-8939d608c6b9 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:48.844307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd5c6c-1dd2-467e-9eee-8939d608c6b9 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:48.844307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd5c6c-1dd2-467e-9eee-8939d608c6b9 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:48.845828 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 771/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 20:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 20:00:48.850516 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb380dc-b617-4188-9390-d6be28d5094a None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:48.851432 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb380dc-b617-4188-9390-d6be28d5094a None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:48.851872 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb380dc-b617-4188-9390-d6be28d5094a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:48.871920 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-9fb380dc-b617-4188-9390-d6be28d5094a None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:00:48.873159 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-9fb380dc-b617-4188-9390-d6be28d5094a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:00:48.881138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 755/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 20:00:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:00:49.033966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd5c6c-1dd2-467e-9eee-8939d608c6b9 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:00:49.049029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd5c6c-1dd2-467e-9eee-8939d608c6b9 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:00:49.054894 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5e6d21-e94b-4833-be37-b6a1caf47c01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:00:49.055503 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e6d21-e94b-4833-be37-b6a1caf47c01 service nova] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:49.055782 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e6d21-e94b-4833-be37-b6a1caf47c01 service nova] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:49.056055 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e6d21-e94b-4833-be37-b6a1caf47c01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:49.064266 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5e6d21-e94b-4833-be37-b6a1caf47c01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:00:49.065872 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5e6d21-e94b-4833-be37-b6a1caf47c01 service nova] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:00:49.075760 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 772/1527] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:00:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:00:53.194880 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff2ded-b7f1-4df7-853f-f7a72cf807e7 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:53.196133 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff2ded-b7f1-4df7-853f-f7a72cf807e7 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:53.196133 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff2ded-b7f1-4df7-853f-f7a72cf807e7 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:53.197658 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 756/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 20:00:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 20:00:53.203077 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a56b68-7387-41dd-8f60-02dfff459eab None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:53.203506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a56b68-7387-41dd-8f60-02dfff459eab None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:53.203626 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a56b68-7387-41dd-8f60-02dfff459eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:53.223867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-65a56b68-7387-41dd-8f60-02dfff459eab None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:00:53.225097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-65a56b68-7387-41dd-8f60-02dfff459eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:00:53.233296 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 773/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 20:00:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:00:53.312258 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff2ded-b7f1-4df7-853f-f7a72cf807e7 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:00:53.314034 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-8dff2ded-b7f1-4df7-853f-f7a72cf807e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:00:53.395408 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff2ded-b7f1-4df7-853f-f7a72cf807e7 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:00:53.400360 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4979e3d8-b80f-439f-9bc5-6a076e405ce5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:00:53.401006 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979e3d8-b80f-439f-9bc5-6a076e405ce5 service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:53.401286 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979e3d8-b80f-439f-9bc5-6a076e405ce5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:53.401558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979e3d8-b80f-439f-9bc5-6a076e405ce5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:53.410760 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4979e3d8-b80f-439f-9bc5-6a076e405ce5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:00:53.412183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4979e3d8-b80f-439f-9bc5-6a076e405ce5 service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:00:53.422074 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 757/1530] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:00:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:00:55.504520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2ed60-6269-499d-9552-b129f8bcf193 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:55.505091 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2ed60-6269-499d-9552-b129f8bcf193 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:55.505091 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2ed60-6269-499d-9552-b129f8bcf193 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:55.506395 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 774/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 20:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 20:00:55.512244 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8d361-cce1-4a30-af34-173170b923ec None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:55.512718 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8d361-cce1-4a30-af34-173170b923ec None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:55.512824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8d361-cce1-4a30-af34-173170b923ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:55.534125 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-64f8d361-cce1-4a30-af34-173170b923ec None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:00:55.535312 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-64f8d361-cce1-4a30-af34-173170b923ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:00:55.543735 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 758/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 20:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:00:55.582892 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2ed60-6269-499d-9552-b129f8bcf193 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:00:55.584571 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-06d2ed60-6269-499d-9552-b129f8bcf193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:00:55.598039 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2ed60-6269-499d-9552-b129f8bcf193 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:00:55.602162 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd61382-6a6b-49bb-aeb6-74be50985759 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:00:55.603092 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd61382-6a6b-49bb-aeb6-74be50985759 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:55.603510 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd61382-6a6b-49bb-aeb6-74be50985759 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:55.603787 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd61382-6a6b-49bb-aeb6-74be50985759 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:55.612863 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd61382-6a6b-49bb-aeb6-74be50985759 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:00:55.614297 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd61382-6a6b-49bb-aeb6-74be50985759 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:00:55.625052 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 775/1533] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:00:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:00:59.549104 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daece2aa-e8b8-44c6-aaf8-04fbfc40d32f None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:59.549627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daece2aa-e8b8-44c6-aaf8-04fbfc40d32f None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:59.549627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daece2aa-e8b8-44c6-aaf8-04fbfc40d32f None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:59.550906 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 759/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 20:00:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 20:00:59.556977 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c88ff3-f5a5-428e-965a-42380a52fa19 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:59.557409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c88ff3-f5a5-428e-965a-42380a52fa19 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:59.557535 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c88ff3-f5a5-428e-965a-42380a52fa19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:59.577392 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-80c88ff3-f5a5-428e-965a-42380a52fa19 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:00:59.578593 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-80c88ff3-f5a5-428e-965a-42380a52fa19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:00:59.586344 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 776/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 20:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:00:59.897878 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daece2aa-e8b8-44c6-aaf8-04fbfc40d32f None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:00:59.899773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-daece2aa-e8b8-44c6-aaf8-04fbfc40d32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:00:59.965388 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:00:59.980076 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daece2aa-e8b8-44c6-aaf8-04fbfc40d32f None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:00:59.983641 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f68fd-8066-4cf6-ad35-4d9b6f89f830 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:00:59.984253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f68fd-8066-4cf6-ad35-4d9b6f89f830 service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:59.984532 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f68fd-8066-4cf6-ad35-4d9b6f89f830 service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:59.984839 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f68fd-8066-4cf6-ad35-4d9b6f89f830 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:59.994533 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90f68fd-8066-4cf6-ad35-4d9b6f89f830 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:00:59.995955 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90f68fd-8066-4cf6-ad35-4d9b6f89f830 service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:01:00.005301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 760/1536] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:00:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:01:02.049290 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617fad1-d4ad-435a-b9b4-5d0337570e5b None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:01:02.049896 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617fad1-d4ad-435a-b9b4-5d0337570e5b None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:01:02.049896 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617fad1-d4ad-435a-b9b4-5d0337570e5b None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:01:02.051192 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 777/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 20:01:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 20:01:02.057483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad350d8e-034f-4ca0-8888-aabada32aaff None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:01:02.057941 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad350d8e-034f-4ca0-8888-aabada32aaff None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:01:02.058053 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad350d8e-034f-4ca0-8888-aabada32aaff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:01:02.078480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-ad350d8e-034f-4ca0-8888-aabada32aaff None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:01:02.079689 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-ad350d8e-034f-4ca0-8888-aabada32aaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:01:02.089093 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 761/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 20:01:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:01:02.389765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f617fad1-d4ad-435a-b9b4-5d0337570e5b None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:01:02.391554 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-f617fad1-d4ad-435a-b9b4-5d0337570e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:01:02.399955 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=84 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:01:02.406547 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f617fad1-d4ad-435a-b9b4-5d0337570e5b None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:01:02.410048 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa60a53-1f12-451a-a096-882623ca322c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:01:02.410652 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa60a53-1f12-451a-a096-882623ca322c service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:01:02.410930 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa60a53-1f12-451a-a096-882623ca322c service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:01:02.411201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa60a53-1f12-451a-a096-882623ca322c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:01:02.421038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa60a53-1f12-451a-a096-882623ca322c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:01:02.422456 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa60a53-1f12-451a-a096-882623ca322c service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:01:02.431775 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 778/1539] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:01:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:01:12.072528 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=11 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:01:12.427595 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=5 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:31.283020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d12c3-2ccf-4c35-882d-b26f149a9b71 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:31.283666 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d12c3-2ccf-4c35-882d-b26f149a9b71 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:31.283666 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d12c3-2ccf-4c35-882d-b26f149a9b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:31.311640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-cc6d12c3-2ccf-4c35-882d-b26f149a9b71 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:31.317236 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-cc6d12c3-2ccf-4c35-882d-b26f149a9b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:31.333867 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 762/1540] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:31.346146 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5a65c-5391-4e82-a110-70af26c791e1 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:31.346731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5a65c-5391-4e82-a110-70af26c791e1 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:31.346731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5a65c-5391-4e82-a110-70af26c791e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:31.377122 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-59c5a65c-5391-4e82-a110-70af26c791e1 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:31.377122 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-59c5a65c-5391-4e82-a110-70af26c791e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:31.390054 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 779/1541] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:31.394909 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d12c3-2ccf-4c35-882d-b26f149a9b71 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:31.410776 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d12c3-2ccf-4c35-882d-b26f149a9b71 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:31.424526 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3eaee-5b25-49ff-80c4-753ad8ac15da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:31.425181 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3eaee-5b25-49ff-80c4-753ad8ac15da admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:31.425461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3eaee-5b25-49ff-80c4-753ad8ac15da admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:31.425730 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3eaee-5b25-49ff-80c4-753ad8ac15da admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:31.426434 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3eaee-5b25-49ff-80c4-753ad8ac15da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:02:31.426737 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3eaee-5b25-49ff-80c4-753ad8ac15da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:31.433851 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3eaee-5b25-49ff-80c4-753ad8ac15da admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:31.448950 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 763/1542] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:02:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:02:31.456898 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5a65c-5391-4e82-a110-70af26c791e1 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:31.467055 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5a65c-5391-4e82-a110-70af26c791e1 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:31.480472 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adaa285-5607-4e2d-9205-c2ae3f514c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:31.481136 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adaa285-5607-4e2d-9205-c2ae3f514c17 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:31.481422 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adaa285-5607-4e2d-9205-c2ae3f514c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:31.481702 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adaa285-5607-4e2d-9205-c2ae3f514c17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:31.482393 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adaa285-5607-4e2d-9205-c2ae3f514c17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:31.484604 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adaa285-5607-4e2d-9205-c2ae3f514c17 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:31.553119 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 780/1543] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:02:31 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:31.557637 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3eaee-5b25-49ff-80c4-753ad8ac15da admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:31.575611 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3eaee-5b25-49ff-80c4-753ad8ac15da admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:31.616302 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795093e-b34b-48e7-a1c9-dbfdb9e350b6 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:31.616798 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795093e-b34b-48e7-a1c9-dbfdb9e350b6 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:31.617084 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795093e-b34b-48e7-a1c9-dbfdb9e350b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:31.652648 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-c795093e-b34b-48e7-a1c9-dbfdb9e350b6 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:31.654309 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-c795093e-b34b-48e7-a1c9-dbfdb9e350b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:31.670010 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae732c-1e18-4a66-89cf-2a75c214a39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:31.670010 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae732c-1e18-4a66-89cf-2a75c214a39e admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:31.670295 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae732c-1e18-4a66-89cf-2a75c214a39e admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:31.670580 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae732c-1e18-4a66-89cf-2a75c214a39e admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:31.672638 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae732c-1e18-4a66-89cf-2a75c214a39e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:31.673340 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae732c-1e18-4a66-89cf-2a75c214a39e admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:31.675946 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 781/1544] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:02:31.677951 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952ef39-d811-4a70-a31f-57e290e16a61 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:31.678576 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952ef39-d811-4a70-a31f-57e290e16a61 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:31.679064 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952ef39-d811-4a70-a31f-57e290e16a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:31.722235 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-0952ef39-d811-4a70-a31f-57e290e16a61 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:31.724721 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-0952ef39-d811-4a70-a31f-57e290e16a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:31.740181 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 764/1545] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:02:31 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:31.741590 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3ba10-a2bf-42c0-a9d0-b24155749032 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:31.742023 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3ba10-a2bf-42c0-a9d0-b24155749032 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:31.742483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3ba10-a2bf-42c0-a9d0-b24155749032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:31.751002 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 782/1546] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:31.752472 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952ef39-d811-4a70-a31f-57e290e16a61 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:31.775386 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952ef39-d811-4a70-a31f-57e290e16a61 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:31.783576 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf21fa2-498f-4bbb-89e7-fc3198ed4eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:31.784692 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf21fa2-498f-4bbb-89e7-fc3198ed4eb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:31.785180 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf21fa2-498f-4bbb-89e7-fc3198ed4eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:31.785686 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf21fa2-498f-4bbb-89e7-fc3198ed4eb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:31.787288 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf21fa2-498f-4bbb-89e7-fc3198ed4eb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:31.789310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf21fa2-498f-4bbb-89e7-fc3198ed4eb9 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:31.790180 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-74c3ba10-a2bf-42c0-a9d0-b24155749032 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:31.796780 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-74c3ba10-a2bf-42c0-a9d0-b24155749032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:31.810815 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 783/1547] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:31.814787 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0b394-1fff-4153-a9a3-336c76bb5eb8 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:31.815324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0b394-1fff-4153-a9a3-336c76bb5eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:31.815806 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0b394-1fff-4153-a9a3-336c76bb5eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:31.818847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 765/1548] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:31.819757 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3ba10-a2bf-42c0-a9d0-b24155749032 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:31.839006 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3ba10-a2bf-42c0-a9d0-b24155749032 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:31.853817 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-b2a0b394-1fff-4153-a9a3-336c76bb5eb8 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:31.855455 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-b2a0b394-1fff-4153-a9a3-336c76bb5eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:31.861245 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a82b0-0bb8-4c9a-895d-a127bff64b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:31.861914 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a82b0-0bb8-4c9a-895d-a127bff64b1b admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:31.862199 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a82b0-0bb8-4c9a-895d-a127bff64b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:31.862474 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a82b0-0bb8-4c9a-895d-a127bff64b1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:31.863189 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0a82b0-0bb8-4c9a-895d-a127bff64b1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:31.866101 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0a82b0-0bb8-4c9a-895d-a127bff64b1b admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:31.874308 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 784/1549] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:31.875689 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0b394-1fff-4153-a9a3-336c76bb5eb8 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:31.885174 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 766/1550] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:31.886164 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a82b0-0bb8-4c9a-895d-a127bff64b1b admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:31.891235 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0b394-1fff-4153-a9a3-336c76bb5eb8 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:31.904956 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aeb2e7-3b8a-4119-8e11-e9f212aefc2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:31.905991 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aeb2e7-3b8a-4119-8e11-e9f212aefc2a admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:31.906507 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aeb2e7-3b8a-4119-8e11-e9f212aefc2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:31.906994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aeb2e7-3b8a-4119-8e11-e9f212aefc2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:31.907694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a82b0-0bb8-4c9a-895d-a127bff64b1b admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:31.908258 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aeb2e7-3b8a-4119-8e11-e9f212aefc2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:02:31.908820 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aeb2e7-3b8a-4119-8e11-e9f212aefc2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:31.910085 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: WARNING py.warnings [None req-e5aeb2e7-3b8a-4119-8e11-e9f212aefc2a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 05 20:02:31.910085 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: warnings.warn(msg) Aug 05 20:02:31.910085 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: Aug 05 20:02:31.911492 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aeb2e7-3b8a-4119-8e11-e9f212aefc2a admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:31.926362 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c5a5a-174a-457b-9082-6c433f88aa0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:31.927513 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c5a5a-174a-457b-9082-6c433f88aa0f admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:31.927941 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c5a5a-174a-457b-9082-6c433f88aa0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:31.928279 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c5a5a-174a-457b-9082-6c433f88aa0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:31.929310 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776c5a5a-174a-457b-9082-6c433f88aa0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:02:31.929725 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776c5a5a-174a-457b-9082-6c433f88aa0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:31.931427 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776c5a5a-174a-457b-9082-6c433f88aa0f admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:31.935616 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 785/1551] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:02:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:02:31.937111 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aeb2e7-3b8a-4119-8e11-e9f212aefc2a admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:31.955238 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aeb2e7-3b8a-4119-8e11-e9f212aefc2a admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:31.956126 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 767/1552] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:02:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:02:31.956991 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c5a5a-174a-457b-9082-6c433f88aa0f admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:31.966866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12bc0cf-8c51-44c2-a37d-9ae2008fe711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:31.967980 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12bc0cf-8c51-44c2-a37d-9ae2008fe711 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:31.968466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12bc0cf-8c51-44c2-a37d-9ae2008fe711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:31.969047 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12bc0cf-8c51-44c2-a37d-9ae2008fe711 admin admin] PATH_INFO: `/v3/projects/14fabce3f6c44a1a915271ac06967e6c/users/36190049a17141deac9faff37e681794/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:31.976769 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c5a5a-174a-457b-9082-6c433f88aa0f admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:31.991640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6533bd71-5a99-483a-bad3-4c677d4957ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:31.992426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533bd71-5a99-483a-bad3-4c677d4957ab admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:31.992734 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533bd71-5a99-483a-bad3-4c677d4957ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:31.993009 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533bd71-5a99-483a-bad3-4c677d4957ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:31.996599 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6533bd71-5a99-483a-bad3-4c677d4957ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:31.996599 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: WARNING py.warnings [None req-6533bd71-5a99-483a-bad3-4c677d4957ab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 20:02:31.996599 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: warnings.warn(msg) Aug 05 20:02:31.996599 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: Aug 05 20:02:31.996599 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6533bd71-5a99-483a-bad3-4c677d4957ab admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:32.000207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12bc0cf-8c51-44c2-a37d-9ae2008fe711 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14fabce3f6c44a1a915271ac06967e6c, user_id=36190049a17141deac9faff37e681794, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:32.003299 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12bc0cf-8c51-44c2-a37d-9ae2008fe711 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:32.029564 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 786/1553] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:02:31 2022] PUT /identity/v3/projects/14fabce3f6c44a1a915271ac06967e6c/users/36190049a17141deac9faff37e681794/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:32.032176 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12bc0cf-8c51-44c2-a37d-9ae2008fe711 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:32.048099 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 768/1554] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:02:31 2022] POST /identity/v3/projects => generated 385 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:32.052516 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6533bd71-5a99-483a-bad3-4c677d4957ab admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:32.179004 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12bc0cf-8c51-44c2-a37d-9ae2008fe711 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:32.184151 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1469a783-f9e1-4e29-91a4-f11cdb579adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:32.184807 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469a783-f9e1-4e29-91a4-f11cdb579adc admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:32.185087 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469a783-f9e1-4e29-91a4-f11cdb579adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:32.185357 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469a783-f9e1-4e29-91a4-f11cdb579adc admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:32.186021 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1469a783-f9e1-4e29-91a4-f11cdb579adc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:32.188251 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1469a783-f9e1-4e29-91a4-f11cdb579adc admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:32.188376 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6533bd71-5a99-483a-bad3-4c677d4957ab admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:32.193790 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f606f-15c9-4884-ab9e-93f779b41631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:32.194492 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f606f-15c9-4884-ab9e-93f779b41631 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:32.194848 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f606f-15c9-4884-ab9e-93f779b41631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:32.195277 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f606f-15c9-4884-ab9e-93f779b41631 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:32.195970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9f606f-15c9-4884-ab9e-93f779b41631 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:32.198110 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9f606f-15c9-4884-ab9e-93f779b41631 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:32.219835 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 787/1555] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:02:32 2022] POST /identity/v3/projects => generated 393 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:32.221123 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5886ecb-d4ef-47a2-9e0a-12c4f849e490 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:32.221404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5886ecb-d4ef-47a2-9e0a-12c4f849e490 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:32.221673 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5886ecb-d4ef-47a2-9e0a-12c4f849e490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:32.230635 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 769/1556] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:02:32 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:02:32.232043 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a08358-e342-4471-a3d6-a9c36de0e251 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:32.232330 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a08358-e342-4471-a3d6-a9c36de0e251 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:32.233418 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a08358-e342-4471-a3d6-a9c36de0e251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:32.246691 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:32.252900 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-48a08358-e342-4471-a3d6-a9c36de0e251 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:32.253228 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-f5886ecb-d4ef-47a2-9e0a-12c4f849e490 None None] MFA Rules not processed for user `36190049a17141deac9faff37e681794`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:32.254481 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-f5886ecb-d4ef-47a2-9e0a-12c4f849e490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:32.255519 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-48a08358-e342-4471-a3d6-a9c36de0e251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:32.284983 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:32.317391 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 770/1557] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:32.319060 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665654e9-0d5b-4f5e-890e-a5ddce4a67e7 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:32.319353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665654e9-0d5b-4f5e-890e-a5ddce4a67e7 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:32.319746 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665654e9-0d5b-4f5e-890e-a5ddce4a67e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:32.324071 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 788/1558] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:32.325476 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e6787-a447-4654-b5b0-4443290255e8 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:32.325763 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e6787-a447-4654-b5b0-4443290255e8 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:32.326031 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e6787-a447-4654-b5b0-4443290255e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:32.347338 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-665654e9-0d5b-4f5e-890e-a5ddce4a67e7 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:32.347932 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-5f2e6787-a447-4654-b5b0-4443290255e8 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:32.348552 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-665654e9-0d5b-4f5e-890e-a5ddce4a67e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:32.349333 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5f2e6787-a447-4654-b5b0-4443290255e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:32.357324 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 771/1559] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:32.358529 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a847890-2cca-4f74-a1fd-1f5e05d69349 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:32.358815 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a847890-2cca-4f74-a1fd-1f5e05d69349 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:32.359092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a847890-2cca-4f74-a1fd-1f5e05d69349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:32.360730 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 789/1560] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:32.362491 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cd4d8-a261-4515-a878-226c0373eb71 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:32.363101 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cd4d8-a261-4515-a878-226c0373eb71 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:32.363779 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cd4d8-a261-4515-a878-226c0373eb71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:32.381871 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-0a847890-2cca-4f74-a1fd-1f5e05d69349 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:32.383080 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-0a847890-2cca-4f74-a1fd-1f5e05d69349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:32.390597 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-9e5cd4d8-a261-4515-a878-226c0373eb71 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:32.391044 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 772/1561] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:32.391763 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-9e5cd4d8-a261-4515-a878-226c0373eb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:32.397329 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a847890-2cca-4f74-a1fd-1f5e05d69349 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:32.400409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 790/1562] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:32.401125 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5cd4d8-a261-4515-a878-226c0373eb71 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:32.408569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a847890-2cca-4f74-a1fd-1f5e05d69349 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:32.410934 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5cd4d8-a261-4515-a878-226c0373eb71 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:32.414042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a67f363-11ad-4e7a-aed6-0c897792e426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:32.414799 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67f363-11ad-4e7a-aed6-0c897792e426 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:32.415080 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67f363-11ad-4e7a-aed6-0c897792e426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:32.415362 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67f363-11ad-4e7a-aed6-0c897792e426 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:32.415882 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247930c4-a203-48ae-8e25-d194d2358e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:32.416029 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a67f363-11ad-4e7a-aed6-0c897792e426 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:32.416488 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247930c4-a203-48ae-8e25-d194d2358e87 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:32.416802 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247930c4-a203-48ae-8e25-d194d2358e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:32.417076 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247930c4-a203-48ae-8e25-d194d2358e87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:32.417769 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247930c4-a203-48ae-8e25-d194d2358e87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:32.417994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a67f363-11ad-4e7a-aed6-0c897792e426 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:32.419344 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247930c4-a203-48ae-8e25-d194d2358e87 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:32.430352 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 791/1563] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:32.431489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b333c7-2d9b-4016-bd3b-0ab85982885c None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:32.431829 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b333c7-2d9b-4016-bd3b-0ab85982885c None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:32.432132 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b333c7-2d9b-4016-bd3b-0ab85982885c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:32.446030 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 773/1564] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:02:32 2022] POST /identity/v3/users => generated 297 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:32.446030 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c08cd-db57-4d89-9f50-290f73f72210 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:32.446030 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c08cd-db57-4d89-9f50-290f73f72210 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:32.446030 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c08cd-db57-4d89-9f50-290f73f72210 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:32.448842 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 774/1565] 10.222.0.39 () {54 vars in 962 bytes} [Fri Aug 5 20:02:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 20:02:32.450001 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a39f8e-0664-4c92-a6e8-b9214a0bb2e3 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:32.450608 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a39f8e-0664-4c92-a6e8-b9214a0bb2e3 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:32.450957 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a39f8e-0664-4c92-a6e8-b9214a0bb2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:32.451948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-18b333c7-2d9b-4016-bd3b-0ab85982885c None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:32.453169 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-18b333c7-2d9b-4016-bd3b-0ab85982885c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:32.461607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 792/1566] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:32.462799 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e6f57-7732-4e93-ba59-32f3c05883d0 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:32.463080 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e6f57-7732-4e93-ba59-32f3c05883d0 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:32.463354 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e6f57-7732-4e93-ba59-32f3c05883d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:32.472692 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-f9a39f8e-0664-4c92-a6e8-b9214a0bb2e3 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:32.473971 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-f9a39f8e-0664-4c92-a6e8-b9214a0bb2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:32.482385 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 775/1567] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:32.482546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-f21e6f57-7732-4e93-ba59-32f3c05883d0 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:32.483349 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a6878-e529-4b3a-9def-d72934f8f3e3 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:32.483465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-f21e6f57-7732-4e93-ba59-32f3c05883d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:32.483635 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a6878-e529-4b3a-9def-d72934f8f3e3 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:32.483908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a6878-e529-4b3a-9def-d72934f8f3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:32.491839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 793/1568] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:32.493964 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207d362-9d10-4729-91ed-555e0aff5db3 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:32.493964 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207d362-9d10-4729-91ed-555e0aff5db3 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:32.493964 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207d362-9d10-4729-91ed-555e0aff5db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:32.504243 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-3d9a6878-e529-4b3a-9def-d72934f8f3e3 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:32.506037 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-3d9a6878-e529-4b3a-9def-d72934f8f3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:32.514371 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-6207d362-9d10-4729-91ed-555e0aff5db3 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:32.515400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 776/1569] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:32.517062 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a6878-e529-4b3a-9def-d72934f8f3e3 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:32.517261 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-6207d362-9d10-4729-91ed-555e0aff5db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:32.527079 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 794/1570] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:32.528540 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207d362-9d10-4729-91ed-555e0aff5db3 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:32.532257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a6878-e529-4b3a-9def-d72934f8f3e3 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:32.536894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92789bb7-2ac9-48b8-ac8a-94dd760cd9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:32.537503 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92789bb7-2ac9-48b8-ac8a-94dd760cd9af admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:32.537782 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92789bb7-2ac9-48b8-ac8a-94dd760cd9af admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:32.538048 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92789bb7-2ac9-48b8-ac8a-94dd760cd9af admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:32.538252 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207d362-9d10-4729-91ed-555e0aff5db3 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:32.538740 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92789bb7-2ac9-48b8-ac8a-94dd760cd9af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:32.540312 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92789bb7-2ac9-48b8-ac8a-94dd760cd9af admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:32.543001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c4981-75fe-4c41-bae8-045526fe8fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:32.543604 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c4981-75fe-4c41-bae8-045526fe8fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:32.543889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c4981-75fe-4c41-bae8-045526fe8fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:32.544169 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c4981-75fe-4c41-bae8-045526fe8fe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:32.544896 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c4981-75fe-4c41-bae8-045526fe8fe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:32.546482 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c4981-75fe-4c41-bae8-045526fe8fe8 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:32.550482 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 777/1571] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:32.551642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628f319-6d75-46a2-99cd-1d58482d8add None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:32.552496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628f319-6d75-46a2-99cd-1d58482d8add None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:32.552800 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628f319-6d75-46a2-99cd-1d58482d8add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:32.559301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 795/1572] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:32.560031 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c4981-75fe-4c41-bae8-045526fe8fe8 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:32.570267 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c4981-75fe-4c41-bae8-045526fe8fe8 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:32.574716 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2a924-7b12-4360-8732-48531a0ffbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:32.575311 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2a924-7b12-4360-8732-48531a0ffbc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:32.575588 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2a924-7b12-4360-8732-48531a0ffbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:32.575858 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2a924-7b12-4360-8732-48531a0ffbc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:32.576532 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f2a924-7b12-4360-8732-48531a0ffbc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:02:32.576867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f2a924-7b12-4360-8732-48531a0ffbc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:32.578448 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f2a924-7b12-4360-8732-48531a0ffbc3 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:32.591221 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 796/1573] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:02:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:02:32.592200 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2a924-7b12-4360-8732-48531a0ffbc3 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:32.593703 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-1628f319-6d75-46a2-99cd-1d58482d8add None None] MFA Rules not processed for user `95e9f085e4f04d118373b8abff27543a`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:32.594896 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-1628f319-6d75-46a2-99cd-1d58482d8add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:32.602814 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2a924-7b12-4360-8732-48531a0ffbc3 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:32.607587 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0e5ba-c985-4264-89aa-cbd0c26737de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:32.608192 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0e5ba-c985-4264-89aa-cbd0c26737de admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:32.608467 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0e5ba-c985-4264-89aa-cbd0c26737de admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:32.608770 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0e5ba-c985-4264-89aa-cbd0c26737de admin admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:32.609458 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a0e5ba-c985-4264-89aa-cbd0c26737de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:02:32.609759 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a0e5ba-c985-4264-89aa-cbd0c26737de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:32.611339 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a0e5ba-c985-4264-89aa-cbd0c26737de admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:32.622882 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 797/1574] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:02:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:02:32.623769 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0e5ba-c985-4264-89aa-cbd0c26737de admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:32.633785 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0e5ba-c985-4264-89aa-cbd0c26737de admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:32.638353 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a7c9e-865f-4299-91e4-ba71b6b406cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:32.638949 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a7c9e-865f-4299-91e4-ba71b6b406cc admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:32.639224 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a7c9e-865f-4299-91e4-ba71b6b406cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:32.639496 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a7c9e-865f-4299-91e4-ba71b6b406cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:32.640167 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951a7c9e-865f-4299-91e4-ba71b6b406cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:02:32.640468 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951a7c9e-865f-4299-91e4-ba71b6b406cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:32.642150 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951a7c9e-865f-4299-91e4-ba71b6b406cc admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:32.655170 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 798/1575] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:02:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:02:32.655992 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a7c9e-865f-4299-91e4-ba71b6b406cc admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:32.666652 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a7c9e-865f-4299-91e4-ba71b6b406cc admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:32.671428 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d5a0cf-6548-43da-847e-4d3c5c467ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:32.672057 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5a0cf-6548-43da-847e-4d3c5c467ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:32.672341 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5a0cf-6548-43da-847e-4d3c5c467ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:32.672639 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5a0cf-6548-43da-847e-4d3c5c467ca6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:32.673328 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d5a0cf-6548-43da-847e-4d3c5c467ca6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:02:32.673635 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d5a0cf-6548-43da-847e-4d3c5c467ca6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:32.675224 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d5a0cf-6548-43da-847e-4d3c5c467ca6 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:32.675872 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 778/1576] 10.222.0.39 () {58 vars in 1059 bytes} [Fri Aug 5 20:02:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:32.676879 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1628f319-6d75-46a2-99cd-1d58482d8add None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:32.687339 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1628f319-6d75-46a2-99cd-1d58482d8add None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:32.694949 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 799/1577] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:02:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:02:32.694949 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d5a0cf-6548-43da-847e-4d3c5c467ca6 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:32.699332 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d5a0cf-6548-43da-847e-4d3c5c467ca6 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:32.699702 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba2bd5-00f3-4d56-8fd8-463382a8f299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:32.700336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba2bd5-00f3-4d56-8fd8-463382a8f299 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:32.700707 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba2bd5-00f3-4d56-8fd8-463382a8f299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:32.701134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba2bd5-00f3-4d56-8fd8-463382a8f299 admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:32.703046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba2bd5-00f3-4d56-8fd8-463382a8f299 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:02:32.703355 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba2bd5-00f3-4d56-8fd8-463382a8f299 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:32.704074 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7855e2b-50d3-4b9a-8fcb-0bee40229d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:32.704786 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7855e2b-50d3-4b9a-8fcb-0bee40229d7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:32.705068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7855e2b-50d3-4b9a-8fcb-0bee40229d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:32.705170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba2bd5-00f3-4d56-8fd8-463382a8f299 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:32.705344 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7855e2b-50d3-4b9a-8fcb-0bee40229d7c admin admin] PATH_INFO: `/v3/projects/5b265913f9bd46b9b36c18d567d0cf66/users/1afe13c8fbb1468b8ef3d0d0c671208d/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:32.720569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 779/1578] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:02:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:02:32.721309 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7855e2b-50d3-4b9a-8fcb-0bee40229d7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b265913f9bd46b9b36c18d567d0cf66, user_id=1afe13c8fbb1468b8ef3d0d0c671208d, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:32.721650 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba2bd5-00f3-4d56-8fd8-463382a8f299 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:32.724535 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7855e2b-50d3-4b9a-8fcb-0bee40229d7c admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:32.732797 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba2bd5-00f3-4d56-8fd8-463382a8f299 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:32.737644 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde6886f-6a70-49ee-91e8-05c17e83206c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:32.738245 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6886f-6a70-49ee-91e8-05c17e83206c admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:32.738521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6886f-6a70-49ee-91e8-05c17e83206c admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:32.738791 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6886f-6a70-49ee-91e8-05c17e83206c admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:32.739503 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde6886f-6a70-49ee-91e8-05c17e83206c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:32.740320 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 800/1579] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:02:32 2022] PUT /identity/v3/projects/5b265913f9bd46b9b36c18d567d0cf66/users/1afe13c8fbb1468b8ef3d0d0c671208d/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:32.741190 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7855e2b-50d3-4b9a-8fcb-0bee40229d7c admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:32.741321 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde6886f-6a70-49ee-91e8-05c17e83206c admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:32.755777 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 780/1580] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:32.755777 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde6886f-6a70-49ee-91e8-05c17e83206c admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:32.763165 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde6886f-6a70-49ee-91e8-05c17e83206c admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:32.766706 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:32.804778 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:32.810451 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7855e2b-50d3-4b9a-8fcb-0bee40229d7c admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:32.816097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4b3591-0f88-4f84-8864-f09893f7a7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:32.816743 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b3591-0f88-4f84-8864-f09893f7a7f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:32.817028 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b3591-0f88-4f84-8864-f09893f7a7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:32.817305 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b3591-0f88-4f84-8864-f09893f7a7f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:32.818508 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4b3591-0f88-4f84-8864-f09893f7a7f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:32.820421 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4b3591-0f88-4f84-8864-f09893f7a7f2 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:32.826053 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6397c10e-26bd-48b7-baba-739e8f933f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:32.826673 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397c10e-26bd-48b7-baba-739e8f933f1b admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:32.826955 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397c10e-26bd-48b7-baba-739e8f933f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:32.827231 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397c10e-26bd-48b7-baba-739e8f933f1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:32.827895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6397c10e-26bd-48b7-baba-739e8f933f1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:32.829857 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6397c10e-26bd-48b7-baba-739e8f933f1b admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:32.851295 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 801/1581] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:02:32 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:32.851295 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4b3591-0f88-4f84-8864-f09893f7a7f2 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:32.868814 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 781/1582] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:02:32 2022] POST /identity/v3/projects => generated 365 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:32.868814 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6397c10e-26bd-48b7-baba-739e8f933f1b admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:32.921849 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4b3591-0f88-4f84-8864-f09893f7a7f2 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:32.936914 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6397c10e-26bd-48b7-baba-739e8f933f1b admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:32.942599 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4232e0-2d0c-4050-9308-de55cae756aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:32.943153 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4232e0-2d0c-4050-9308-de55cae756aa service nova] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:32.943437 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4232e0-2d0c-4050-9308-de55cae756aa service nova] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:32.943717 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4232e0-2d0c-4050-9308-de55cae756aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:32.979874 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27cc24-af40-43d0-9a43-34fbf89665f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:32.980494 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27cc24-af40-43d0-9a43-34fbf89665f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:32.980808 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27cc24-af40-43d0-9a43-34fbf89665f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:32.981085 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27cc24-af40-43d0-9a43-34fbf89665f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:32.981753 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27cc24-af40-43d0-9a43-34fbf89665f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:32.983689 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27cc24-af40-43d0-9a43-34fbf89665f6 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:33.002188 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4232e0-2d0c-4050-9308-de55cae756aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:33.003847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4232e0-2d0c-4050-9308-de55cae756aa service nova] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:33.011830 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 802/1583] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:02:32 2022] POST /identity/v3/projects => generated 361 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:33.012735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27cc24-af40-43d0-9a43-34fbf89665f6 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:33.013505 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 782/1584] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:33.014296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4232e0-2d0c-4050-9308-de55cae756aa service nova] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:33.025409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4232e0-2d0c-4050-9308-de55cae756aa service nova] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:33.058020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:33.081323 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27cc24-af40-43d0-9a43-34fbf89665f6 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:33.086838 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f15d27f-e368-4dc0-a4fb-60a4e5206d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:33.087450 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f15d27f-e368-4dc0-a4fb-60a4e5206d47 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:33.087730 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f15d27f-e368-4dc0-a4fb-60a4e5206d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:33.088053 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f15d27f-e368-4dc0-a4fb-60a4e5206d47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:33.088757 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f15d27f-e368-4dc0-a4fb-60a4e5206d47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:33.090693 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f15d27f-e368-4dc0-a4fb-60a4e5206d47 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:33.095344 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059d956-9763-49c6-8eac-e5a2bd61f249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:33.096130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059d956-9763-49c6-8eac-e5a2bd61f249 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:33.096319 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059d956-9763-49c6-8eac-e5a2bd61f249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:33.096620 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059d956-9763-49c6-8eac-e5a2bd61f249 admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:33.097422 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c059d956-9763-49c6-8eac-e5a2bd61f249 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:33.100310 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c059d956-9763-49c6-8eac-e5a2bd61f249 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:33.117055 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:33.129976 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 803/1585] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:02:33 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:33.131731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8cf1ac-14cf-4eef-b80a-0d1e1e4f92b7 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:33.132023 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8cf1ac-14cf-4eef-b80a-0d1e1e4f92b7 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:33.132300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8cf1ac-14cf-4eef-b80a-0d1e1e4f92b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:33.137700 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 783/1586] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:02:33 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:33.138777 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059d956-9763-49c6-8eac-e5a2bd61f249 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:33.159456 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-7a8cf1ac-14cf-4eef-b80a-0d1e1e4f92b7 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:33.160709 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-7a8cf1ac-14cf-4eef-b80a-0d1e1e4f92b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:33.214777 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059d956-9763-49c6-8eac-e5a2bd61f249 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:33.220012 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d58141-4bc3-4ecd-8509-aedebf68c996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:33.220704 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d58141-4bc3-4ecd-8509-aedebf68c996 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:33.221032 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d58141-4bc3-4ecd-8509-aedebf68c996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:33.221265 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d58141-4bc3-4ecd-8509-aedebf68c996 admin admin] PATH_INFO: `/v3/projects/c5b5718607784ec9be3289f7916f70a6/users/293b8a62f19c45c1bf5b897c7cf40edb/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:33.228474 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 804/1587] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:33.229354 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8cf1ac-14cf-4eef-b80a-0d1e1e4f92b7 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:33.244438 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8cf1ac-14cf-4eef-b80a-0d1e1e4f92b7 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:33.246603 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d58141-4bc3-4ecd-8509-aedebf68c996 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5b5718607784ec9be3289f7916f70a6, user_id=293b8a62f19c45c1bf5b897c7cf40edb, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:33.247180 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: WARNING py.warnings [None req-f4d58141-4bc3-4ecd-8509-aedebf68c996 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 20:02:33.247180 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: warnings.warn(msg) Aug 05 20:02:33.247180 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: Aug 05 20:02:33.249427 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d58141-4bc3-4ecd-8509-aedebf68c996 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:33.250016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42c4c8-862a-4ddc-8387-7b97a0456b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:33.250573 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42c4c8-862a-4ddc-8387-7b97a0456b51 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:33.250959 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42c4c8-862a-4ddc-8387-7b97a0456b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:33.251303 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42c4c8-862a-4ddc-8387-7b97a0456b51 admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:33.252449 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42c4c8-862a-4ddc-8387-7b97a0456b51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:33.254451 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: WARNING py.warnings [None req-1f42c4c8-862a-4ddc-8387-7b97a0456b51 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 20:02:33.254451 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: warnings.warn(msg) Aug 05 20:02:33.254451 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: Aug 05 20:02:33.256071 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42c4c8-862a-4ddc-8387-7b97a0456b51 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:33.266713 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 784/1588] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:02:33 2022] PUT /identity/v3/projects/c5b5718607784ec9be3289f7916f70a6/users/293b8a62f19c45c1bf5b897c7cf40edb/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:33.268189 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d58141-4bc3-4ecd-8509-aedebf68c996 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:33.283923 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 805/1589] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:02:33 2022] POST /identity/v3/users => generated 290 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:33.283923 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42c4c8-862a-4ddc-8387-7b97a0456b51 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:33.293035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42c4c8-862a-4ddc-8387-7b97a0456b51 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:33.346515 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d58141-4bc3-4ecd-8509-aedebf68c996 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:33.352811 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa6f68-fb43-4420-9447-3310bbb2c61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:33.353799 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa6f68-fb43-4420-9447-3310bbb2c61c admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:33.354212 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa6f68-fb43-4420-9447-3310bbb2c61c admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:33.354497 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa6f68-fb43-4420-9447-3310bbb2c61c admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:33.355225 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa6f68-fb43-4420-9447-3310bbb2c61c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:33.356038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbb247-6952-404b-98e8-654bfdb9a06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:33.356760 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbb247-6952-404b-98e8-654bfdb9a06d admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:33.357118 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbb247-6952-404b-98e8-654bfdb9a06d admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:33.357455 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa6f68-fb43-4420-9447-3310bbb2c61c admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:33.357577 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbb247-6952-404b-98e8-654bfdb9a06d admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:33.358253 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbb247-6952-404b-98e8-654bfdb9a06d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:33.360271 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbb247-6952-404b-98e8-654bfdb9a06d admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:33.385422 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 785/1590] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:02:33 2022] POST /identity/v3/users => generated 283 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:33.386350 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa6f68-fb43-4420-9447-3310bbb2c61c admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:33.388776 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 806/1591] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:02:33 2022] POST /identity/v3/users => generated 281 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:33.389677 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbb247-6952-404b-98e8-654bfdb9a06d admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:33.398902 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa6f68-fb43-4420-9447-3310bbb2c61c admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:33.401397 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbb247-6952-404b-98e8-654bfdb9a06d admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:33.403831 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018dbd97-f3ca-486c-a2ff-f106f78b576a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:33.404637 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018dbd97-f3ca-486c-a2ff-f106f78b576a admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:33.404919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018dbd97-f3ca-486c-a2ff-f106f78b576a admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:33.405191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018dbd97-f3ca-486c-a2ff-f106f78b576a admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:33.405846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018dbd97-f3ca-486c-a2ff-f106f78b576a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:33.406637 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4644b13e-b0b0-4448-a852-bea0de201b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:33.407304 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644b13e-b0b0-4448-a852-bea0de201b9a admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:33.407655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644b13e-b0b0-4448-a852-bea0de201b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:33.407814 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018dbd97-f3ca-486c-a2ff-f106f78b576a admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:33.408066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644b13e-b0b0-4448-a852-bea0de201b9a admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:33.408830 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4644b13e-b0b0-4448-a852-bea0de201b9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:33.410994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4644b13e-b0b0-4448-a852-bea0de201b9a admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:33.434088 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 807/1592] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:02:33 2022] POST /identity/v3/users => generated 289 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:33.434419 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 786/1593] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:02:33 2022] POST /identity/v3/users => generated 287 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:33.439513 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4644b13e-b0b0-4448-a852-bea0de201b9a admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:33.439651 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df91da5-f5fa-4d83-ab51-2a4161a1aec5 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:33.439651 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df91da5-f5fa-4d83-ab51-2a4161a1aec5 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:33.439651 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df91da5-f5fa-4d83-ab51-2a4161a1aec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:33.462429 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-4df91da5-f5fa-4d83-ab51-2a4161a1aec5 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:33.463638 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-4df91da5-f5fa-4d83-ab51-2a4161a1aec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:33.475143 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 787/1594] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:33.475143 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df91da5-f5fa-4d83-ab51-2a4161a1aec5 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:33.478126 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:33.491630 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df91da5-f5fa-4d83-ab51-2a4161a1aec5 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:33.508730 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad5579-d7a0-4675-93aa-ed621b45bfd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:33.508730 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad5579-d7a0-4675-93aa-ed621b45bfd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:33.508730 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad5579-d7a0-4675-93aa-ed621b45bfd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:33.508730 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad5579-d7a0-4675-93aa-ed621b45bfd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:33.509506 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bad5579-d7a0-4675-93aa-ed621b45bfd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:33.510536 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bad5579-d7a0-4675-93aa-ed621b45bfd3 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:33.511933 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4644b13e-b0b0-4448-a852-bea0de201b9a admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:33.516077 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65929952-e2d8-4543-80f9-9b3a34990a1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:33.517330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65929952-e2d8-4543-80f9-9b3a34990a1b service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:33.517617 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65929952-e2d8-4543-80f9-9b3a34990a1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:33.517891 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65929952-e2d8-4543-80f9-9b3a34990a1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:33.526288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 788/1595] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:33.526288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad5579-d7a0-4675-93aa-ed621b45bfd3 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:33.529986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65929952-e2d8-4543-80f9-9b3a34990a1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:33.531545 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65929952-e2d8-4543-80f9-9b3a34990a1b service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:33.535982 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:33.540784 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad5579-d7a0-4675-93aa-ed621b45bfd3 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:33.543902 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 808/1596] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:33.544904 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65929952-e2d8-4543-80f9-9b3a34990a1b service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:33.549625 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59654e7-5bd0-4763-b177-fe6077a42ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:33.550486 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59654e7-5bd0-4763-b177-fe6077a42ebf admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:33.550920 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59654e7-5bd0-4763-b177-fe6077a42ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:33.551321 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59654e7-5bd0-4763-b177-fe6077a42ebf admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:33.552226 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59654e7-5bd0-4763-b177-fe6077a42ebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:33.553993 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59654e7-5bd0-4763-b177-fe6077a42ebf admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:33.559562 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65929952-e2d8-4543-80f9-9b3a34990a1b service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:33.565175 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6fe4e-1b60-4180-a3d5-9e515a735060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:33.565327 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 789/1597] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:33.566126 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59654e7-5bd0-4763-b177-fe6077a42ebf admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:33.566684 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6fe4e-1b60-4180-a3d5-9e515a735060 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:33.567064 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6fe4e-1b60-4180-a3d5-9e515a735060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:33.568190 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6fe4e-1b60-4180-a3d5-9e515a735060 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:33.569330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a6fe4e-1b60-4180-a3d5-9e515a735060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:33.571024 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a6fe4e-1b60-4180-a3d5-9e515a735060 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:33.579574 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59654e7-5bd0-4763-b177-fe6077a42ebf admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:33.582914 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 809/1598] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:33.584891 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6fe4e-1b60-4180-a3d5-9e515a735060 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:33.588122 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b62d0d-3501-497e-a1d5-bc48538d2cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:33.593952 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b62d0d-3501-497e-a1d5-bc48538d2cff admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:33.594233 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b62d0d-3501-497e-a1d5-bc48538d2cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:33.594508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b62d0d-3501-497e-a1d5-bc48538d2cff admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:33.595218 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b62d0d-3501-497e-a1d5-bc48538d2cff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:33.595614 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6fe4e-1b60-4180-a3d5-9e515a735060 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:33.596901 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b62d0d-3501-497e-a1d5-bc48538d2cff admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:33.601011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0348271-8de4-4ee3-9841-e793f243a2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:33.601702 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0348271-8de4-4ee3-9841-e793f243a2a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:33.602049 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0348271-8de4-4ee3-9841-e793f243a2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:33.602392 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0348271-8de4-4ee3-9841-e793f243a2a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:33.603171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0348271-8de4-4ee3-9841-e793f243a2a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:33.604866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0348271-8de4-4ee3-9841-e793f243a2a6 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:33.612334 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 790/1599] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:33.612334 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b62d0d-3501-497e-a1d5-bc48538d2cff admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:33.620281 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 810/1600] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:33.621189 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0348271-8de4-4ee3-9841-e793f243a2a6 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:33.624293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b62d0d-3501-497e-a1d5-bc48538d2cff admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:33.628566 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab563ce1-759f-4df7-aa67-0b68e675027e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:33.629302 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab563ce1-759f-4df7-aa67-0b68e675027e service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:33.629628 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab563ce1-759f-4df7-aa67-0b68e675027e service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:33.630740 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab563ce1-759f-4df7-aa67-0b68e675027e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:33.635035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0348271-8de4-4ee3-9841-e793f243a2a6 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:33.641189 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c2d4b-7200-4624-863e-b966e4b34314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:33.641803 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c2d4b-7200-4624-863e-b966e4b34314 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:33.642087 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c2d4b-7200-4624-863e-b966e4b34314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:33.642354 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c2d4b-7200-4624-863e-b966e4b34314 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:33.643038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c2d4b-7200-4624-863e-b966e4b34314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:33.643207 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab563ce1-759f-4df7-aa67-0b68e675027e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:33.644779 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c2d4b-7200-4624-863e-b966e4b34314 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:33.645390 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab563ce1-759f-4df7-aa67-0b68e675027e service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:33.657245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 811/1601] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:33.657245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74bcc5-e951-4b82-8cf3-4fa41a50b8b3 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:33.657245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74bcc5-e951-4b82-8cf3-4fa41a50b8b3 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:33.657245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74bcc5-e951-4b82-8cf3-4fa41a50b8b3 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:33.661691 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 791/1602] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:33.666120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 812/1603] 10.222.0.39 () {54 vars in 962 bytes} [Fri Aug 5 20:02:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 20:02:33.666295 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab563ce1-759f-4df7-aa67-0b68e675027e service neutron] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:33.666406 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74bcc5-e951-4b82-8cf3-4fa41a50b8b3 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:33.679994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74bcc5-e951-4b82-8cf3-4fa41a50b8b3 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:33.681632 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab563ce1-759f-4df7-aa67-0b68e675027e service neutron] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:33.685333 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188cc807-c64d-4f8c-8974-cf53a884d947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:33.685878 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188cc807-c64d-4f8c-8974-cf53a884d947 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:33.686165 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188cc807-c64d-4f8c-8974-cf53a884d947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:33.686442 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188cc807-c64d-4f8c-8974-cf53a884d947 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:33.686963 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfad055-2e8c-479f-b869-053d002b5bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:33.687293 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188cc807-c64d-4f8c-8974-cf53a884d947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:33.687716 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfad055-2e8c-479f-b869-053d002b5bbe admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:33.688074 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfad055-2e8c-479f-b869-053d002b5bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:33.688412 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfad055-2e8c-479f-b869-053d002b5bbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:33.689083 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188cc807-c64d-4f8c-8974-cf53a884d947 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:33.689210 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfad055-2e8c-479f-b869-053d002b5bbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:33.690872 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfad055-2e8c-479f-b869-053d002b5bbe admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:33.709907 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 813/1604] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:33.710130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 792/1605] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:33.710230 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188cc807-c64d-4f8c-8974-cf53a884d947 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:33.710336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfad055-2e8c-479f-b869-053d002b5bbe admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:33.718270 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfad055-2e8c-479f-b869-053d002b5bbe admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:33.720967 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188cc807-c64d-4f8c-8974-cf53a884d947 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:33.723403 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a7a59-1cb4-4985-91ca-cc41930c9d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:33.724666 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a7a59-1cb4-4985-91ca-cc41930c9d6f admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:33.725012 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a7a59-1cb4-4985-91ca-cc41930c9d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:33.725443 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a7a59-1cb4-4985-91ca-cc41930c9d6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:33.726192 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a7a59-1cb4-4985-91ca-cc41930c9d6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:33.727915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a7a59-1cb4-4985-91ca-cc41930c9d6f admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:33.730066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a5ba4-2c77-4364-9cae-20a318ae3b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:33.731225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a5ba4-2c77-4364-9cae-20a318ae3b4a admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:33.731739 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a5ba4-2c77-4364-9cae-20a318ae3b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:33.732062 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a5ba4-2c77-4364-9cae-20a318ae3b4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:33.732845 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813a5ba4-2c77-4364-9cae-20a318ae3b4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:33.735239 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813a5ba4-2c77-4364-9cae-20a318ae3b4a admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:33.740304 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 793/1606] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:33.741111 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a7a59-1cb4-4985-91ca-cc41930c9d6f admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:33.753434 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a7a59-1cb4-4985-91ca-cc41930c9d6f admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:33.758375 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 814/1607] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:33.758375 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d939c5-5347-4ce9-b565-4011343815f7 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:33.758375 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d939c5-5347-4ce9-b565-4011343815f7 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:33.758691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d939c5-5347-4ce9-b565-4011343815f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:33.759953 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419dbbca-3c0f-4d4c-ac5d-146db4cb24e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:33.760753 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419dbbca-3c0f-4d4c-ac5d-146db4cb24e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:33.761114 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419dbbca-3c0f-4d4c-ac5d-146db4cb24e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:33.761438 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419dbbca-3c0f-4d4c-ac5d-146db4cb24e0 admin admin] PATH_INFO: `/v3/projects/fbd1312a3fb2498488a4525c15e26f2d/users/4da8ef6d2bc141c79a068a8f9a6a3254/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:33.778674 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419dbbca-3c0f-4d4c-ac5d-146db4cb24e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbd1312a3fb2498488a4525c15e26f2d, user_id=4da8ef6d2bc141c79a068a8f9a6a3254, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:33.781561 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419dbbca-3c0f-4d4c-ac5d-146db4cb24e0 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:33.785224 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-e3d939c5-5347-4ce9-b565-4011343815f7 None None] MFA Rules not processed for user `95e9f085e4f04d118373b8abff27543a`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:33.787855 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e3d939c5-5347-4ce9-b565-4011343815f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:33.797952 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 794/1608] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:02:33 2022] PUT /identity/v3/projects/fbd1312a3fb2498488a4525c15e26f2d/users/4da8ef6d2bc141c79a068a8f9a6a3254/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:33.805504 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419dbbca-3c0f-4d4c-ac5d-146db4cb24e0 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:33.875792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 815/1609] 10.222.0.39 () {58 vars in 1059 bytes} [Fri Aug 5 20:02:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:33.876402 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d939c5-5347-4ce9-b565-4011343815f7 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:33.889007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d939c5-5347-4ce9-b565-4011343815f7 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:33.898193 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419dbbca-3c0f-4d4c-ac5d-146db4cb24e0 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:33.899107 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad10982-b11e-4fc6-aca2-f979a79d841c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:33.899856 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad10982-b11e-4fc6-aca2-f979a79d841c admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:33.900138 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad10982-b11e-4fc6-aca2-f979a79d841c admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:33.900440 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad10982-b11e-4fc6-aca2-f979a79d841c admin admin] PATH_INFO: `/v3/projects/a78bb9942fcc4ef896ee6f8eee6b4935/users/2243e27d5cec4ea7a5376d438a83f745/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:33.904918 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b56ad-fdca-4f38-9dd1-22ae23b1c3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:33.905557 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b56ad-fdca-4f38-9dd1-22ae23b1c3a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:33.905933 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b56ad-fdca-4f38-9dd1-22ae23b1c3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:33.906092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b56ad-fdca-4f38-9dd1-22ae23b1c3a5 admin admin] PATH_INFO: `/v3/projects/de7f4f5adc804c17834f22ab4fb0c29d/users/f12b6003a1824329949e3f1ed6b43c29/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:33.918133 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad10982-b11e-4fc6-aca2-f979a79d841c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a78bb9942fcc4ef896ee6f8eee6b4935, user_id=2243e27d5cec4ea7a5376d438a83f745, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:33.920862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad10982-b11e-4fc6-aca2-f979a79d841c admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:33.926071 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b56ad-fdca-4f38-9dd1-22ae23b1c3a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de7f4f5adc804c17834f22ab4fb0c29d, user_id=f12b6003a1824329949e3f1ed6b43c29, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:33.929640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b56ad-fdca-4f38-9dd1-22ae23b1c3a5 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:33.938228 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 816/1610] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:02:33 2022] PUT /identity/v3/projects/a78bb9942fcc4ef896ee6f8eee6b4935/users/2243e27d5cec4ea7a5376d438a83f745/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:33.939360 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad10982-b11e-4fc6-aca2-f979a79d841c admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:33.954482 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 795/1611] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:02:33 2022] PUT /identity/v3/projects/de7f4f5adc804c17834f22ab4fb0c29d/users/f12b6003a1824329949e3f1ed6b43c29/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:33.955627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b56ad-fdca-4f38-9dd1-22ae23b1c3a5 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:34.017214 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:34.031734 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad10982-b11e-4fc6-aca2-f979a79d841c admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:34.032092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b56ad-fdca-4f38-9dd1-22ae23b1c3a5 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:34.038654 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259e5ff-2967-4b19-b998-bfa4fe07c589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:34.039623 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259e5ff-2967-4b19-b998-bfa4fe07c589 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:34.040130 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259e5ff-2967-4b19-b998-bfa4fe07c589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:34.040672 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259e5ff-2967-4b19-b998-bfa4fe07c589 admin admin] PATH_INFO: `/v3/projects/2bb000ac20014d618784b262288bac32/users/83e975aed96648fc9c06f95d49c6f6f9/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:34.041121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c986179-ce57-4cf8-baff-b058c416c17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:34.041919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c986179-ce57-4cf8-baff-b058c416c17c admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:34.042314 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c986179-ce57-4cf8-baff-b058c416c17c admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:34.042724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c986179-ce57-4cf8-baff-b058c416c17c admin admin] PATH_INFO: `/v3/projects/64ec77769e724c6b8f54f8c64e3f6dca/users/aebfc9902d2646e5ab5b197d7b2681be/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:34.065741 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f259e5ff-2967-4b19-b998-bfa4fe07c589 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bb000ac20014d618784b262288bac32, user_id=83e975aed96648fc9c06f95d49c6f6f9, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:34.065920 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c986179-ce57-4cf8-baff-b058c416c17c admin admin] RBAC: Authorizing `identity:create_grant(project_id=64ec77769e724c6b8f54f8c64e3f6dca, user_id=aebfc9902d2646e5ab5b197d7b2681be, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:34.068957 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c986179-ce57-4cf8-baff-b058c416c17c admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:34.070049 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f259e5ff-2967-4b19-b998-bfa4fe07c589 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:34.073719 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:34.094955 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 796/1612] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:02:33 2022] PUT /identity/v3/projects/64ec77769e724c6b8f54f8c64e3f6dca/users/aebfc9902d2646e5ab5b197d7b2681be/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:34.097205 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f0b4d-9b13-4a09-9714-eae13ff28ece None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:34.097770 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f0b4d-9b13-4a09-9714-eae13ff28ece None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:34.097770 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f0b4d-9b13-4a09-9714-eae13ff28ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:34.105862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 817/1613] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:02:33 2022] PUT /identity/v3/projects/2bb000ac20014d618784b262288bac32/users/83e975aed96648fc9c06f95d49c6f6f9/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:34.109296 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d92ed-b112-4529-97be-69af88d19a2b None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:34.109739 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d92ed-b112-4529-97be-69af88d19a2b None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:34.110025 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d92ed-b112-4529-97be-69af88d19a2b None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:34.115165 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 818/1614] 10.222.0.39 () {54 vars in 912 bytes} [Fri Aug 5 20:02:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 20:02:34.115165 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf5069-1a5b-4111-b24e-9540dce78c53 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:34.116101 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf5069-1a5b-4111-b24e-9540dce78c53 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:34.116628 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf5069-1a5b-4111-b24e-9540dce78c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:34.170013 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-4d0f0b4d-9b13-4a09-9714-eae13ff28ece None None] MFA Rules not processed for user `4da8ef6d2bc141c79a068a8f9a6a3254`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:34.171778 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-4d0f0b4d-9b13-4a09-9714-eae13ff28ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:34.179853 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-11bf5069-1a5b-4111-b24e-9540dce78c53 None None] MFA Rules not processed for user `2243e27d5cec4ea7a5376d438a83f745`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:34.181337 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-11bf5069-1a5b-4111-b24e-9540dce78c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:34.279253 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 819/1615] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:34.279253 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf9a06-751d-48b8-92b9-017af9a0736e None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:34.279253 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf9a06-751d-48b8-92b9-017af9a0736e None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:34.279253 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf9a06-751d-48b8-92b9-017af9a0736e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:34.288195 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 797/1616] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:34.289640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ce0fa-f48c-466d-b5c6-179e08c90110 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:34.289984 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ce0fa-f48c-466d-b5c6-179e08c90110 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:34.290273 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ce0fa-f48c-466d-b5c6-179e08c90110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:34.311409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-8dbf9a06-751d-48b8-92b9-017af9a0736e None None] MFA Rules not processed for user `f12b6003a1824329949e3f1ed6b43c29`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:34.312764 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-8dbf9a06-751d-48b8-92b9-017af9a0736e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:34.326571 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-b66ce0fa-f48c-466d-b5c6-179e08c90110 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:34.327784 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-b66ce0fa-f48c-466d-b5c6-179e08c90110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:34.397917 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 798/1617] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:34.399365 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcc50d-203f-4675-83da-6c3efb955464 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:34.399596 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcc50d-203f-4675-83da-6c3efb955464 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:34.399878 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcc50d-203f-4675-83da-6c3efb955464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:34.422297 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 820/1618] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:34.423638 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a04219-2f30-48f0-94ef-93ab51f3639c None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:34.423924 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a04219-2f30-48f0-94ef-93ab51f3639c None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:34.424199 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a04219-2f30-48f0-94ef-93ab51f3639c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:34.437976 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-6dfcc50d-203f-4675-83da-6c3efb955464 None None] MFA Rules not processed for user `aebfc9902d2646e5ab5b197d7b2681be`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:34.440248 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-6dfcc50d-203f-4675-83da-6c3efb955464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:34.445866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-72a04219-2f30-48f0-94ef-93ab51f3639c None None] MFA Rules not processed for user `95e9f085e4f04d118373b8abff27543a`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:34.447066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-72a04219-2f30-48f0-94ef-93ab51f3639c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:34.509446 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:34.522219 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 799/1620] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:34.522415 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 821/1620] 10.222.0.39 () {58 vars in 1009 bytes} [Fri Aug 5 20:02:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:34.523103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfcc50d-203f-4675-83da-6c3efb955464 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:34.523386 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a04219-2f30-48f0-94ef-93ab51f3639c None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:34.530502 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:34.538589 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfcc50d-203f-4675-83da-6c3efb955464 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:34.539245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a04219-2f30-48f0-94ef-93ab51f3639c None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:34.547356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c51bdb-1625-4809-a45b-f89344b3dc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:34.547925 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d8fbe3-ee07-4604-9c48-363fc2b8cfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:34.548985 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8fbe3-ee07-4604-9c48-363fc2b8cfd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:34.549256 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c51bdb-1625-4809-a45b-f89344b3dc24 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:34.549491 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8fbe3-ee07-4604-9c48-363fc2b8cfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:34.549949 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c51bdb-1625-4809-a45b-f89344b3dc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:34.550079 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8fbe3-ee07-4604-9c48-363fc2b8cfd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:34.550582 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c51bdb-1625-4809-a45b-f89344b3dc24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:34.552776 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d8fbe3-ee07-4604-9c48-363fc2b8cfd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:34.553106 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c51bdb-1625-4809-a45b-f89344b3dc24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:34.556419 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d8fbe3-ee07-4604-9c48-363fc2b8cfd1 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:34.557372 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c51bdb-1625-4809-a45b-f89344b3dc24 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:34.587786 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 822/1621] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:02:34 2022] POST /identity/v3/projects => generated 379 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:34.588691 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 800/1622] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:02:34 2022] POST /identity/v3/projects => generated 363 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:34.588828 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c51bdb-1625-4809-a45b-f89344b3dc24 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:34.589806 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d8fbe3-ee07-4604-9c48-363fc2b8cfd1 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:34.655845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d8fbe3-ee07-4604-9c48-363fc2b8cfd1 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:34.665227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed255e4-7a64-4137-b0ef-f54a0181647e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:34.665868 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed255e4-7a64-4137-b0ef-f54a0181647e service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:34.666619 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed255e4-7a64-4137-b0ef-f54a0181647e service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:34.666834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c51bdb-1625-4809-a45b-f89344b3dc24 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:34.666945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed255e4-7a64-4137-b0ef-f54a0181647e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:34.725662 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe38bc-ad3d-42c2-9d29-905b89c475e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:34.726372 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe38bc-ad3d-42c2-9d29-905b89c475e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:34.726742 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe38bc-ad3d-42c2-9d29-905b89c475e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:34.727305 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe38bc-ad3d-42c2-9d29-905b89c475e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:34.739672 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed255e4-7a64-4137-b0ef-f54a0181647e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:34.741494 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed255e4-7a64-4137-b0ef-f54a0181647e service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:34.755430 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 801/1623] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:34.756707 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed255e4-7a64-4137-b0ef-f54a0181647e service cinder] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:34.876793 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed255e4-7a64-4137-b0ef-f54a0181647e service cinder] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:34.880264 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe38bc-ad3d-42c2-9d29-905b89c475e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:34.883360 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fe46aa-a1de-417a-a369-ef90787d565f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:34.884319 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe46aa-a1de-417a-a369-ef90787d565f admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:34.884743 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe46aa-a1de-417a-a369-ef90787d565f admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:34.885087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe46aa-a1de-417a-a369-ef90787d565f admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:34.886202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fe46aa-a1de-417a-a369-ef90787d565f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:34.886678 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe38bc-ad3d-42c2-9d29-905b89c475e4 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:34.888443 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fe46aa-a1de-417a-a369-ef90787d565f admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:34.915778 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 823/1624] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:34.921187 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe38bc-ad3d-42c2-9d29-905b89c475e4 service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:34.932089 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 802/1625] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:02:34 2022] POST /identity/v3/projects => generated 373 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:34.935473 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fe46aa-a1de-417a-a369-ef90787d565f admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:34.942009 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe38bc-ad3d-42c2-9d29-905b89c475e4 service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:34.952858 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:35.030279 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:35.038128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111bf8f-72e6-4d3a-a90a-7eccbeb6c9f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:35.039133 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111bf8f-72e6-4d3a-a90a-7eccbeb6c9f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:35.039512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111bf8f-72e6-4d3a-a90a-7eccbeb6c9f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:35.040210 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111bf8f-72e6-4d3a-a90a-7eccbeb6c9f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:35.081274 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fe46aa-a1de-417a-a369-ef90787d565f admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:35.094474 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba34a1-452e-456e-bcc4-e56a39ae1bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:35.095822 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba34a1-452e-456e-bcc4-e56a39ae1bb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:35.097894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba34a1-452e-456e-bcc4-e56a39ae1bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:35.098293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba34a1-452e-456e-bcc4-e56a39ae1bb1 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:35.100871 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba34a1-452e-456e-bcc4-e56a39ae1bb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:35.102748 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba34a1-452e-456e-bcc4-e56a39ae1bb1 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:35.134216 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5111bf8f-72e6-4d3a-a90a-7eccbeb6c9f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:35.135653 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5111bf8f-72e6-4d3a-a90a-7eccbeb6c9f3 service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:35.138569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 803/1626] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:02:34 2022] POST /identity/v3/users => generated 296 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:35.145217 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba34a1-452e-456e-bcc4-e56a39ae1bb1 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:35.153653 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 824/1627] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:35.154026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111bf8f-72e6-4d3a-a90a-7eccbeb6c9f3 service cinder] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:35.163619 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba34a1-452e-456e-bcc4-e56a39ae1bb1 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:35.165608 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111bf8f-72e6-4d3a-a90a-7eccbeb6c9f3 service cinder] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:35.169864 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72167fa5-3c4e-486c-b5c9-658765a2d7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:35.171004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72167fa5-3c4e-486c-b5c9-658765a2d7f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:35.171235 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8785d2cd-6e74-4109-90cb-1f9a402d3c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:35.171631 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72167fa5-3c4e-486c-b5c9-658765a2d7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:35.171973 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785d2cd-6e74-4109-90cb-1f9a402d3c05 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:35.172297 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72167fa5-3c4e-486c-b5c9-658765a2d7f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:35.172431 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785d2cd-6e74-4109-90cb-1f9a402d3c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:35.172735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785d2cd-6e74-4109-90cb-1f9a402d3c05 admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:35.173464 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8785d2cd-6e74-4109-90cb-1f9a402d3c05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:35.173903 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72167fa5-3c4e-486c-b5c9-658765a2d7f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:35.175750 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8785d2cd-6e74-4109-90cb-1f9a402d3c05 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:35.177615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72167fa5-3c4e-486c-b5c9-658765a2d7f0 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:35.204459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 825/1628] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:02:35 2022] POST /identity/v3/users => generated 293 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:35.204459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8785d2cd-6e74-4109-90cb-1f9a402d3c05 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:35.213412 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 804/1629] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:02:35 2022] POST /identity/v3/users => generated 288 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:35.215180 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72167fa5-3c4e-486c-b5c9-658765a2d7f0 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:35.217164 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8785d2cd-6e74-4109-90cb-1f9a402d3c05 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:35.222473 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96cb0c8-68f9-438c-9ab9-d798bd98df63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:35.223213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cb0c8-68f9-438c-9ab9-d798bd98df63 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:35.224194 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cb0c8-68f9-438c-9ab9-d798bd98df63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:35.224563 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cb0c8-68f9-438c-9ab9-d798bd98df63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:35.225400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96cb0c8-68f9-438c-9ab9-d798bd98df63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:35.227224 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96cb0c8-68f9-438c-9ab9-d798bd98df63 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:35.227376 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72167fa5-3c4e-486c-b5c9-658765a2d7f0 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:35.233511 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ec318-75d1-4571-9495-50be5182d8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:35.234503 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ec318-75d1-4571-9495-50be5182d8e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:35.234872 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ec318-75d1-4571-9495-50be5182d8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:35.235287 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ec318-75d1-4571-9495-50be5182d8e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:35.236207 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ec318-75d1-4571-9495-50be5182d8e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:35.238585 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ec318-75d1-4571-9495-50be5182d8e4 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:35.244380 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 826/1630] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:35.245488 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96cb0c8-68f9-438c-9ab9-d798bd98df63 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:35.255460 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 805/1631] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:35.256487 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ec318-75d1-4571-9495-50be5182d8e4 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:35.257165 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96cb0c8-68f9-438c-9ab9-d798bd98df63 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:35.267566 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ec318-75d1-4571-9495-50be5182d8e4 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:35.270759 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ce8b8-2a69-49ce-8570-095d4e39e6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:35.271966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ce8b8-2a69-49ce-8570-095d4e39e6ed admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:35.272358 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ce8b8-2a69-49ce-8570-095d4e39e6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:35.272763 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ce8b8-2a69-49ce-8570-095d4e39e6ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:35.273586 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895ce8b8-2a69-49ce-8570-095d4e39e6ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:35.276093 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d925935-4128-4392-8774-5fe9ba9a4b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:35.277086 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d925935-4128-4392-8774-5fe9ba9a4b64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:35.277420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d925935-4128-4392-8774-5fe9ba9a4b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:35.278029 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d925935-4128-4392-8774-5fe9ba9a4b64 admin admin] PATH_INFO: `/v3/projects/a945deeae8dd4371aab957d526ffe1f7/users/a62039722920453b8cc6f8befc0aaa45/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:35.278985 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895ce8b8-2a69-49ce-8570-095d4e39e6ed admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:35.299673 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d925935-4128-4392-8774-5fe9ba9a4b64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a945deeae8dd4371aab957d526ffe1f7, user_id=a62039722920453b8cc6f8befc0aaa45, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:35.303443 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d925935-4128-4392-8774-5fe9ba9a4b64 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:35.306290 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 827/1632] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:35.307823 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ce8b8-2a69-49ce-8570-095d4e39e6ed admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:35.320994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ce8b8-2a69-49ce-8570-095d4e39e6ed admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:35.333476 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83012bc-3602-41e3-a656-789322d103a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:35.334105 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83012bc-3602-41e3-a656-789322d103a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:35.334477 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83012bc-3602-41e3-a656-789322d103a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:35.334899 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83012bc-3602-41e3-a656-789322d103a2 admin admin] PATH_INFO: `/v3/projects/dd20569bd4c1444f81cfb700f5d1e20a/users/564c041d8a0e4d9990f9010a4d8db400/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:35.350245 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 806/1633] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:02:35 2022] PUT /identity/v3/projects/a945deeae8dd4371aab957d526ffe1f7/users/a62039722920453b8cc6f8befc0aaa45/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:35.351081 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d925935-4128-4392-8774-5fe9ba9a4b64 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:35.366170 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83012bc-3602-41e3-a656-789322d103a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd20569bd4c1444f81cfb700f5d1e20a, user_id=564c041d8a0e4d9990f9010a4d8db400, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:35.369516 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83012bc-3602-41e3-a656-789322d103a2 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:35.394877 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 828/1634] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:02:35 2022] PUT /identity/v3/projects/dd20569bd4c1444f81cfb700f5d1e20a/users/564c041d8a0e4d9990f9010a4d8db400/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:35.394877 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83012bc-3602-41e3-a656-789322d103a2 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:35.476765 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d925935-4128-4392-8774-5fe9ba9a4b64 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:35.525126 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83012bc-3602-41e3-a656-789322d103a2 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:35.533748 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a697f-c536-4bc6-8cbb-39bcc42e04e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:35.534554 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a697f-c536-4bc6-8cbb-39bcc42e04e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:35.534934 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a697f-c536-4bc6-8cbb-39bcc42e04e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:35.536242 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a697f-c536-4bc6-8cbb-39bcc42e04e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:35.536242 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a697f-c536-4bc6-8cbb-39bcc42e04e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:35.537940 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a697f-c536-4bc6-8cbb-39bcc42e04e6 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:35.559468 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 829/1635] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:35.560739 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a697f-c536-4bc6-8cbb-39bcc42e04e6 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:35.574112 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a697f-c536-4bc6-8cbb-39bcc42e04e6 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:35.581214 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4fc0fc-f980-4989-9133-e53e7ae75d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:35.582062 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4fc0fc-f980-4989-9133-e53e7ae75d3c admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:35.582569 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4fc0fc-f980-4989-9133-e53e7ae75d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:35.583205 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4fc0fc-f980-4989-9133-e53e7ae75d3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:35.584256 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4fc0fc-f980-4989-9133-e53e7ae75d3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:35.587711 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4fc0fc-f980-4989-9133-e53e7ae75d3c admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:35.606836 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:35.620463 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f28461-3840-437d-a817-908282d15b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:35.622360 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f28461-3840-437d-a817-908282d15b46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:35.623008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 830/1636] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:35.624538 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01860ed1-6730-4953-95e8-eddd6468d7a5 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:35.624853 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01860ed1-6730-4953-95e8-eddd6468d7a5 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:35.625819 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01860ed1-6730-4953-95e8-eddd6468d7a5 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:35.628852 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f28461-3840-437d-a817-908282d15b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:35.629311 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f28461-3840-437d-a817-908282d15b46 admin admin] PATH_INFO: `/v3/projects/32a5ef819ffd4760acd1a9cc88afe165/users/effe6d1e7d8e47a5a5921f683c0b9444/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:35.641718 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 831/1637] 10.222.0.39 () {54 vars in 912 bytes} [Fri Aug 5 20:02:35 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 20:02:35.642564 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01860ed1-6730-4953-95e8-eddd6468d7a5 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:35.666184 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f28461-3840-437d-a817-908282d15b46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32a5ef819ffd4760acd1a9cc88afe165, user_id=effe6d1e7d8e47a5a5921f683c0b9444, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:35.670121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f28461-3840-437d-a817-908282d15b46 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:35.696201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01860ed1-6730-4953-95e8-eddd6468d7a5 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:35.704301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 807/1638] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:02:35 2022] PUT /identity/v3/projects/32a5ef819ffd4760acd1a9cc88afe165/users/effe6d1e7d8e47a5a5921f683c0b9444/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:35.714867 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f28461-3840-437d-a817-908282d15b46 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:35.725093 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:35.847539 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968f8b8-d111-4ff1-b7c9-ac39aad25b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:35.849066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968f8b8-d111-4ff1-b7c9-ac39aad25b8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:35.849696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968f8b8-d111-4ff1-b7c9-ac39aad25b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:35.850071 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f28461-3840-437d-a817-908282d15b46 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:35.850429 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968f8b8-d111-4ff1-b7c9-ac39aad25b8e admin admin] PATH_INFO: `/v3/projects/a945deeae8dd4371aab957d526ffe1f7/users/a62039722920453b8cc6f8befc0aaa45/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:35.854581 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a968f8b8-d111-4ff1-b7c9-ac39aad25b8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a945deeae8dd4371aab957d526ffe1f7, user_id=a62039722920453b8cc6f8befc0aaa45, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:35.857385 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4cb218-72e3-4389-a68a-55d5ffba7a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:35.858310 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4cb218-72e3-4389-a68a-55d5ffba7a7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:35.858839 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4cb218-72e3-4389-a68a-55d5ffba7a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:35.859212 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4cb218-72e3-4389-a68a-55d5ffba7a7f admin admin] PATH_INFO: `/v3/projects/dd20569bd4c1444f81cfb700f5d1e20a/users/564c041d8a0e4d9990f9010a4d8db400/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:35.862365 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4cb218-72e3-4389-a68a-55d5ffba7a7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd20569bd4c1444f81cfb700f5d1e20a, user_id=564c041d8a0e4d9990f9010a4d8db400, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:35.862768 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a968f8b8-d111-4ff1-b7c9-ac39aad25b8e admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:35.865878 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4cb218-72e3-4389-a68a-55d5ffba7a7f admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:35.891250 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 808/1639] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:02:35 2022] PUT /identity/v3/projects/dd20569bd4c1444f81cfb700f5d1e20a/users/564c041d8a0e4d9990f9010a4d8db400/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:35.891250 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12029bc5-72ee-4981-aaea-fc17ca611158 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:35.891250 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12029bc5-72ee-4981-aaea-fc17ca611158 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:35.891250 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12029bc5-72ee-4981-aaea-fc17ca611158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:35.899559 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 832/1640] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:02:35 2022] PUT /identity/v3/projects/a945deeae8dd4371aab957d526ffe1f7/users/a62039722920453b8cc6f8befc0aaa45/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:35.901638 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968f8b8-d111-4ff1-b7c9-ac39aad25b8e admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:35.943769 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-12029bc5-72ee-4981-aaea-fc17ca611158 None None] MFA Rules not processed for user `95e9f085e4f04d118373b8abff27543a`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:35.945054 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-12029bc5-72ee-4981-aaea-fc17ca611158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:36.064308 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968f8b8-d111-4ff1-b7c9-ac39aad25b8e admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:36.071885 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00bcd8-cd67-473f-bf02-4bcef26be26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:36.072509 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00bcd8-cd67-473f-bf02-4bcef26be26b admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:36.072830 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00bcd8-cd67-473f-bf02-4bcef26be26b admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:36.073404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00bcd8-cd67-473f-bf02-4bcef26be26b admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:36.074609 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e00bcd8-cd67-473f-bf02-4bcef26be26b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:36.077122 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e00bcd8-cd67-473f-bf02-4bcef26be26b admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:36.096552 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 833/1641] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:36.098125 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06bdad7-1ea3-4e6a-8e5a-73de2c6d004a None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:36.098423 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06bdad7-1ea3-4e6a-8e5a-73de2c6d004a None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:36.099296 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06bdad7-1ea3-4e6a-8e5a-73de2c6d004a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:36.103433 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 809/1642] 10.222.0.39 () {58 vars in 1009 bytes} [Fri Aug 5 20:02:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:36.105046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae638d30-3e5b-4b04-9eeb-38e59881db02 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:36.105336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae638d30-3e5b-4b04-9eeb-38e59881db02 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:36.105613 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae638d30-3e5b-4b04-9eeb-38e59881db02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:36.157951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-ae638d30-3e5b-4b04-9eeb-38e59881db02 None None] MFA Rules not processed for user `a62039722920453b8cc6f8befc0aaa45`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:36.159296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-ae638d30-3e5b-4b04-9eeb-38e59881db02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:36.172191 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-e06bdad7-1ea3-4e6a-8e5a-73de2c6d004a None None] MFA Rules not processed for user `564c041d8a0e4d9990f9010a4d8db400`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:36.174224 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e06bdad7-1ea3-4e6a-8e5a-73de2c6d004a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:36.248179 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:36.294003 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 810/1643] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:36 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:36.295299 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae638d30-3e5b-4b04-9eeb-38e59881db02 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:36.295695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 834/1644] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:36.296716 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06bdad7-1ea3-4e6a-8e5a-73de2c6d004a None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:36.320693 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae638d30-3e5b-4b04-9eeb-38e59881db02 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:36.326806 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6adbc5-1136-49f9-8ae6-aa7e4f8f29ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:36.328032 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6adbc5-1136-49f9-8ae6-aa7e4f8f29ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:36.328488 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6adbc5-1136-49f9-8ae6-aa7e4f8f29ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:36.328948 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6adbc5-1136-49f9-8ae6-aa7e4f8f29ba admin admin] PATH_INFO: `/v3/projects/32a5ef819ffd4760acd1a9cc88afe165/users/effe6d1e7d8e47a5a5921f683c0b9444/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:36.331976 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6adbc5-1136-49f9-8ae6-aa7e4f8f29ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=32a5ef819ffd4760acd1a9cc88afe165, user_id=effe6d1e7d8e47a5a5921f683c0b9444, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:36.334446 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:36.337915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6adbc5-1136-49f9-8ae6-aa7e4f8f29ba admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:36.370469 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 811/1645] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:02:36 2022] PUT /identity/v3/projects/32a5ef819ffd4760acd1a9cc88afe165/users/effe6d1e7d8e47a5a5921f683c0b9444/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:36.371656 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6adbc5-1136-49f9-8ae6-aa7e4f8f29ba admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:36.384248 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06bdad7-1ea3-4e6a-8e5a-73de2c6d004a None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:36.449055 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a56de-b5bc-4903-ad19-c6b9e25ea52b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:36.450134 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a56de-b5bc-4903-ad19-c6b9e25ea52b service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:36.451373 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a56de-b5bc-4903-ad19-c6b9e25ea52b service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:36.451860 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a56de-b5bc-4903-ad19-c6b9e25ea52b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:36.497640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6adbc5-1136-49f9-8ae6-aa7e4f8f29ba admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:36.508494 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa19316-0ac8-49cc-a31a-8039c9fad8c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:36.509234 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa19316-0ac8-49cc-a31a-8039c9fad8c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:36.509514 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa19316-0ac8-49cc-a31a-8039c9fad8c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:36.509783 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa19316-0ac8-49cc-a31a-8039c9fad8c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:36.557563 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a56de-b5bc-4903-ad19-c6b9e25ea52b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:36.559230 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a56de-b5bc-4903-ad19-c6b9e25ea52b service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:36.579222 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 835/1646] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:36.582252 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce0953-1739-4982-b85c-a124c8b3de5c None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:36.596228 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce0953-1739-4982-b85c-a124c8b3de5c None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:36.596593 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce0953-1739-4982-b85c-a124c8b3de5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:36.706512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-30ce0953-1739-4982-b85c-a124c8b3de5c None None] MFA Rules not processed for user `effe6d1e7d8e47a5a5921f683c0b9444`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:36.706512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-30ce0953-1739-4982-b85c-a124c8b3de5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:36.744292 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa19316-0ac8-49cc-a31a-8039c9fad8c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:36.746573 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa19316-0ac8-49cc-a31a-8039c9fad8c5 service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:36.785164 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 812/1647] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:36.785164 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa19316-0ac8-49cc-a31a-8039c9fad8c5 service cinder] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:36.805636 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa19316-0ac8-49cc-a31a-8039c9fad8c5 service cinder] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:36.813362 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3203ac27-aa1c-468c-89a1-900c686b4aff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:36.814289 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203ac27-aa1c-468c-89a1-900c686b4aff service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:36.814566 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203ac27-aa1c-468c-89a1-900c686b4aff service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:36.814838 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203ac27-aa1c-468c-89a1-900c686b4aff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:36.893279 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 836/1648] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:36 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:36.959024 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3203ac27-aa1c-468c-89a1-900c686b4aff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:36.960565 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3203ac27-aa1c-468c-89a1-900c686b4aff service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:36.989029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce0953-1739-4982-b85c-a124c8b3de5c None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:37.018769 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 813/1649] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:36 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:37.019755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:37.091936 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3203ac27-aa1c-468c-89a1-900c686b4aff service cinder] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:37.108444 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3203ac27-aa1c-468c-89a1-900c686b4aff service cinder] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:37.124268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03655ad-1305-460b-ad84-5e1baca8b99e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:37.126846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03655ad-1305-460b-ad84-5e1baca8b99e service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:37.127256 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03655ad-1305-460b-ad84-5e1baca8b99e service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:37.127866 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03655ad-1305-460b-ad84-5e1baca8b99e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:37.145332 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce0953-1739-4982-b85c-a124c8b3de5c None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:37.156827 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e950201-5cfc-4270-bcf2-946f79f453d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:37.157612 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e950201-5cfc-4270-bcf2-946f79f453d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:37.157958 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e950201-5cfc-4270-bcf2-946f79f453d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:37.158250 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e950201-5cfc-4270-bcf2-946f79f453d3 admin admin] PATH_INFO: `/v3/users/36190049a17141deac9faff37e681794` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:37.163117 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e950201-5cfc-4270-bcf2-946f79f453d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36190049a17141deac9faff37e681794)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:37.165233 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e950201-5cfc-4270-bcf2-946f79f453d3 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:37.170433 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03655ad-1305-460b-ad84-5e1baca8b99e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:37.172360 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03655ad-1305-460b-ad84-5e1baca8b99e service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:37.196927 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 814/1650] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:37 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:37.295802 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03655ad-1305-460b-ad84-5e1baca8b99e service cinder] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:37.327177 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03655ad-1305-460b-ad84-5e1baca8b99e service cinder] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:37.332376 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-2e950201-5cfc-4270-bcf2-946f79f453d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36190049a17141deac9faff37e681794'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:37.339813 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be182ad-56d4-4462-bd66-24e53a72885f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:37.340366 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be182ad-56d4-4462-bd66-24e53a72885f service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:37.340671 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be182ad-56d4-4462-bd66-24e53a72885f service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:37.346485 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be182ad-56d4-4462-bd66-24e53a72885f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:37.353417 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-2e950201-5cfc-4270-bcf2-946f79f453d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36190049a17141deac9faff37e681794'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:37.362094 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:37.373724 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-2e950201-5cfc-4270-bcf2-946f79f453d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36190049a17141deac9faff37e681794'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:37.413645 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-2e950201-5cfc-4270-bcf2-946f79f453d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36190049a17141deac9faff37e681794'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:37.415088 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-2e950201-5cfc-4270-bcf2-946f79f453d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36190049a17141deac9faff37e681794'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:37.447657 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 837/1651] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:02:36 2022] DELETE /identity/v3/users/36190049a17141deac9faff37e681794 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:02:37.448628 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b135d-121d-4bc9-89d1-f4c5fbff4916 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:37.448967 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b135d-121d-4bc9-89d1-f4c5fbff4916 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:37.449248 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b135d-121d-4bc9-89d1-f4c5fbff4916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:37.508017 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-bc7b135d-121d-4bc9-89d1-f4c5fbff4916 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:37.508017 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-bc7b135d-121d-4bc9-89d1-f4c5fbff4916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:37.582226 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be182ad-56d4-4462-bd66-24e53a72885f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:37.582226 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be182ad-56d4-4462-bd66-24e53a72885f service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:37.609733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 815/1652] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:37.646730 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be182ad-56d4-4462-bd66-24e53a72885f service cinder] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:37.662305 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be182ad-56d4-4462-bd66-24e53a72885f service cinder] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:37.671604 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 838/1653] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:02:37.774754 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954273bc-b8ba-4adb-b341-d94e92214ed2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:37.776970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954273bc-b8ba-4adb-b341-d94e92214ed2 service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:37.778070 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954273bc-b8ba-4adb-b341-d94e92214ed2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:37.778449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954273bc-b8ba-4adb-b341-d94e92214ed2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:37.783392 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-954273bc-b8ba-4adb-b341-d94e92214ed2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:37.890529 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b135d-121d-4bc9-89d1-f4c5fbff4916 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:37.892235 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-bc7b135d-121d-4bc9-89d1-f4c5fbff4916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:37.969973 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:38.016196 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954273bc-b8ba-4adb-b341-d94e92214ed2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:38.018033 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954273bc-b8ba-4adb-b341-d94e92214ed2 service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:38.040897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b135d-121d-4bc9-89d1-f4c5fbff4916 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:38.045319 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 816/1654] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:38.071914 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088f071b-12a2-4869-89b1-5bbec198b87e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:38.072568 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f071b-12a2-4869-89b1-5bbec198b87e service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:38.079980 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f071b-12a2-4869-89b1-5bbec198b87e service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:38.080227 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f071b-12a2-4869-89b1-5bbec198b87e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:38.117908 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088f071b-12a2-4869-89b1-5bbec198b87e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:38.173208 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088f071b-12a2-4869-89b1-5bbec198b87e service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:38.173208 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 839/1655] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:02:38.843485 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954273bc-b8ba-4adb-b341-d94e92214ed2 service cinder] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:38.847181 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-954273bc-b8ba-4adb-b341-d94e92214ed2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:38.878697 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954273bc-b8ba-4adb-b341-d94e92214ed2 service cinder] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:38.893790 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78b922-9c67-4a68-8ca8-abb28086a130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:38.894984 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78b922-9c67-4a68-8ca8-abb28086a130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:38.897010 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78b922-9c67-4a68-8ca8-abb28086a130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:38.897010 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78b922-9c67-4a68-8ca8-abb28086a130 admin admin] PATH_INFO: `/v3/users/aebfc9902d2646e5ab5b197d7b2681be` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:38.900156 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b78b922-9c67-4a68-8ca8-abb28086a130 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aebfc9902d2646e5ab5b197d7b2681be)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:38.901318 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: WARNING py.warnings [None req-3b78b922-9c67-4a68-8ca8-abb28086a130 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 20:02:38.901318 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: warnings.warn(msg) Aug 05 20:02:38.901318 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: Aug 05 20:02:38.902671 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b78b922-9c67-4a68-8ca8-abb28086a130 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:39.049995 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-3b78b922-9c67-4a68-8ca8-abb28086a130 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aebfc9902d2646e5ab5b197d7b2681be'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:39.078344 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-3b78b922-9c67-4a68-8ca8-abb28086a130 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aebfc9902d2646e5ab5b197d7b2681be'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:39.086205 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:39.135892 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-3b78b922-9c67-4a68-8ca8-abb28086a130 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aebfc9902d2646e5ab5b197d7b2681be'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:39.137671 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-3b78b922-9c67-4a68-8ca8-abb28086a130 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aebfc9902d2646e5ab5b197d7b2681be'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:39.186124 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-3b78b922-9c67-4a68-8ca8-abb28086a130 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aebfc9902d2646e5ab5b197d7b2681be'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:39.210258 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 817/1656] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:02:38 2022] DELETE /identity/v3/users/aebfc9902d2646e5ab5b197d7b2681be => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:39.218066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a5f17-33e0-4ce5-85d0-14831b03c6e9 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:39.218601 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a5f17-33e0-4ce5-85d0-14831b03c6e9 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:39.218601 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a5f17-33e0-4ce5-85d0-14831b03c6e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:39.275499 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-7c2a5f17-33e0-4ce5-85d0-14831b03c6e9 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:39.277061 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-7c2a5f17-33e0-4ce5-85d0-14831b03c6e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:39.405235 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 840/1657] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:39.474634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78b922-9c67-4a68-8ca8-abb28086a130 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:39.476182 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-3b78b922-9c67-4a68-8ca8-abb28086a130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:39.592801 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78b922-9c67-4a68-8ca8-abb28086a130 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:39.601750 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c4fee-0b87-4063-8cf8-af60209ccd52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:39.602585 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c4fee-0b87-4063-8cf8-af60209ccd52 service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:39.602971 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c4fee-0b87-4063-8cf8-af60209ccd52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:39.603352 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c4fee-0b87-4063-8cf8-af60209ccd52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:39.616869 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9c4fee-0b87-4063-8cf8-af60209ccd52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:39.618652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9c4fee-0b87-4063-8cf8-af60209ccd52 service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:39.636962 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 818/1658] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:39.723484 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a5f17-33e0-4ce5-85d0-14831b03c6e9 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:39.725071 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-7c2a5f17-33e0-4ce5-85d0-14831b03c6e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:39.746282 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a5f17-33e0-4ce5-85d0-14831b03c6e9 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:39.758998 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9498f5-478f-465b-9f1d-6984fcf1eb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:39.766959 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9498f5-478f-465b-9f1d-6984fcf1eb1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:39.767757 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9498f5-478f-465b-9f1d-6984fcf1eb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:39.768211 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9498f5-478f-465b-9f1d-6984fcf1eb1b admin admin] PATH_INFO: `/v3/projects/14fabce3f6c44a1a915271ac06967e6c` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:39.769642 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9498f5-478f-465b-9f1d-6984fcf1eb1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=14fabce3f6c44a1a915271ac06967e6c)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:39.771889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: WARNING py.warnings [None req-1b9498f5-478f-465b-9f1d-6984fcf1eb1b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 20:02:39.771889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: warnings.warn(msg) Aug 05 20:02:39.771889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: Aug 05 20:02:39.773191 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9498f5-478f-465b-9f1d-6984fcf1eb1b admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:39.916331 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-1b9498f5-478f-465b-9f1d-6984fcf1eb1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14fabce3f6c44a1a915271ac06967e6c', 'request_id': 'req-1b9498f5-478f-465b-9f1d-6984fcf1eb1b'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:39.960167 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-1b9498f5-478f-465b-9f1d-6984fcf1eb1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14fabce3f6c44a1a915271ac06967e6c', 'request_id': 'req-1b9498f5-478f-465b-9f1d-6984fcf1eb1b'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:39.978748 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-1b9498f5-478f-465b-9f1d-6984fcf1eb1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14fabce3f6c44a1a915271ac06967e6c', 'request_id': 'req-1b9498f5-478f-465b-9f1d-6984fcf1eb1b'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:40.010596 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-1b9498f5-478f-465b-9f1d-6984fcf1eb1b admin admin] The token cache is being invalidate because project 14fabce3f6c44a1a915271ac06967e6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:02:40.011770 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-1b9498f5-478f-465b-9f1d-6984fcf1eb1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:40.013094 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-1b9498f5-478f-465b-9f1d-6984fcf1eb1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:40.022864 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 841/1659] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:02:39 2022] DELETE /identity/v3/projects/14fabce3f6c44a1a915271ac06967e6c => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:40.027683 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa5f49-8788-4d6f-b998-7a47e53f4a38 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:40.028157 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa5f49-8788-4d6f-b998-7a47e53f4a38 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:40.028260 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa5f49-8788-4d6f-b998-7a47e53f4a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:40.092323 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-37aa5f49-8788-4d6f-b998-7a47e53f4a38 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:40.102413 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-37aa5f49-8788-4d6f-b998-7a47e53f4a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:40.286018 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 819/1660] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:40.294283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01aeca-2546-4202-b1a9-0d51af175eae None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:40.294765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01aeca-2546-4202-b1a9-0d51af175eae None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:40.294839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01aeca-2546-4202-b1a9-0d51af175eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:40.344287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-4e01aeca-2546-4202-b1a9-0d51af175eae None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:40.345560 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-4e01aeca-2546-4202-b1a9-0d51af175eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:40.372547 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa5f49-8788-4d6f-b998-7a47e53f4a38 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:40.373096 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 842/1661] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:40.390100 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa5f49-8788-4d6f-b998-7a47e53f4a38 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:40.402208 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e05949-462d-4e9b-94ae-e61373c778ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:40.409591 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e05949-462d-4e9b-94ae-e61373c778ea admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:40.410523 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e05949-462d-4e9b-94ae-e61373c778ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:40.411846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e05949-462d-4e9b-94ae-e61373c778ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:40.414565 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e05949-462d-4e9b-94ae-e61373c778ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:02:40.415338 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e05949-462d-4e9b-94ae-e61373c778ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:40.421894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e05949-462d-4e9b-94ae-e61373c778ea admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:40.461508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 820/1662] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:02:40.462832 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01aeca-2546-4202-b1a9-0d51af175eae None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:40.477781 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01aeca-2546-4202-b1a9-0d51af175eae None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:40.493005 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be073b9-1523-4084-8313-042792a9ed7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:40.500311 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be073b9-1523-4084-8313-042792a9ed7c admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:40.500764 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be073b9-1523-4084-8313-042792a9ed7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:40.501231 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be073b9-1523-4084-8313-042792a9ed7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:40.502032 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be073b9-1523-4084-8313-042792a9ed7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:40.507549 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be073b9-1523-4084-8313-042792a9ed7c admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:40.585538 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 843/1663] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:02:40 2022] POST /identity/v3/projects => generated 405 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:40.587798 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e05949-462d-4e9b-94ae-e61373c778ea admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:40.616704 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e05949-462d-4e9b-94ae-e61373c778ea admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:40.865706 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04742553-2e43-4b19-ad8e-3cd743bcef05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:40.867089 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04742553-2e43-4b19-ad8e-3cd743bcef05 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:40.871990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04742553-2e43-4b19-ad8e-3cd743bcef05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:40.873597 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04742553-2e43-4b19-ad8e-3cd743bcef05 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:40.876504 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04742553-2e43-4b19-ad8e-3cd743bcef05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:40.882904 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04742553-2e43-4b19-ad8e-3cd743bcef05 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:40.920598 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:40.954404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 821/1664] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:02:40 2022] POST /identity/v3/users => generated 303 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:40.959438 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be073b9-1523-4084-8313-042792a9ed7c admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:40.977616 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be073b9-1523-4084-8313-042792a9ed7c admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:40.993972 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5763d419-6511-459c-a6a1-8ad56c92a353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:40.994706 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763d419-6511-459c-a6a1-8ad56c92a353 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:40.995043 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763d419-6511-459c-a6a1-8ad56c92a353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:40.995323 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763d419-6511-459c-a6a1-8ad56c92a353 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:41.000070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5763d419-6511-459c-a6a1-8ad56c92a353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:41.000070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5763d419-6511-459c-a6a1-8ad56c92a353 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:41.014834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 844/1665] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:41.018643 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04742553-2e43-4b19-ad8e-3cd743bcef05 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:41.048114 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04742553-2e43-4b19-ad8e-3cd743bcef05 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:41.064660 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da309d5d-961d-4ab1-a901-8edca57e36a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:41.066623 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da309d5d-961d-4ab1-a901-8edca57e36a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:41.068821 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da309d5d-961d-4ab1-a901-8edca57e36a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:41.069306 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da309d5d-961d-4ab1-a901-8edca57e36a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:41.070580 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da309d5d-961d-4ab1-a901-8edca57e36a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:41.076429 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da309d5d-961d-4ab1-a901-8edca57e36a2 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:41.123015 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 822/1666] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:41.124675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5763d419-6511-459c-a6a1-8ad56c92a353 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:41.151875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5763d419-6511-459c-a6a1-8ad56c92a353 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:41.157507 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888cb06-71f8-4dad-8af7-7998e926bf1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:41.158558 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888cb06-71f8-4dad-8af7-7998e926bf1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:41.159146 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888cb06-71f8-4dad-8af7-7998e926bf1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:41.159253 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888cb06-71f8-4dad-8af7-7998e926bf1b admin admin] PATH_INFO: `/v3/projects/4e11b728ee874e50aeee2fda90fe0c62/users/13a8b01944e241bfb9fd9c2e30c2fc43/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:41.178409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:41.181431 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2888cb06-71f8-4dad-8af7-7998e926bf1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e11b728ee874e50aeee2fda90fe0c62, user_id=13a8b01944e241bfb9fd9c2e30c2fc43, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:41.184456 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2888cb06-71f8-4dad-8af7-7998e926bf1b admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:41.212665 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 845/1667] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:02:41 2022] PUT /identity/v3/projects/4e11b728ee874e50aeee2fda90fe0c62/users/13a8b01944e241bfb9fd9c2e30c2fc43/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:41.217401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd3efa-fcd3-423e-b177-281835f7666b None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:41.217688 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd3efa-fcd3-423e-b177-281835f7666b None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:41.217966 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd3efa-fcd3-423e-b177-281835f7666b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:41.272691 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-ecbd3efa-fcd3-423e-b177-281835f7666b None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:41.278561 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-ecbd3efa-fcd3-423e-b177-281835f7666b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:41.470635 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 823/1668] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:02:41.481992 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888cb06-71f8-4dad-8af7-7998e926bf1b admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:41.485224 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-2888cb06-71f8-4dad-8af7-7998e926bf1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:41.640738 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888cb06-71f8-4dad-8af7-7998e926bf1b admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:41.650874 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b242939c-f7ff-475c-9bf3-5ba0f6faa109 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:41.655721 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242939c-f7ff-475c-9bf3-5ba0f6faa109 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:41.656209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242939c-f7ff-475c-9bf3-5ba0f6faa109 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:41.658470 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242939c-f7ff-475c-9bf3-5ba0f6faa109 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:41.693521 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b242939c-f7ff-475c-9bf3-5ba0f6faa109 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:41.697776 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b242939c-f7ff-475c-9bf3-5ba0f6faa109 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:41.728899 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 846/1669] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:42.090514 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd3efa-fcd3-423e-b177-281835f7666b None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:42.092495 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-ecbd3efa-fcd3-423e-b177-281835f7666b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:42.121198 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd3efa-fcd3-423e-b177-281835f7666b None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:42.149779 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12584d2-233b-4a9d-86ad-a913140f3430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:42.150690 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12584d2-233b-4a9d-86ad-a913140f3430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:42.154716 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12584d2-233b-4a9d-86ad-a913140f3430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:42.156776 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12584d2-233b-4a9d-86ad-a913140f3430 admin admin] PATH_INFO: `/v3/projects/64ec77769e724c6b8f54f8c64e3f6dca` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:42.162508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12584d2-233b-4a9d-86ad-a913140f3430 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64ec77769e724c6b8f54f8c64e3f6dca)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:42.170633 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12584d2-233b-4a9d-86ad-a913140f3430 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:42.300625 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-b12584d2-233b-4a9d-86ad-a913140f3430 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64ec77769e724c6b8f54f8c64e3f6dca', 'request_id': 'req-b12584d2-233b-4a9d-86ad-a913140f3430'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:42.325569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-b12584d2-233b-4a9d-86ad-a913140f3430 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64ec77769e724c6b8f54f8c64e3f6dca', 'request_id': 'req-b12584d2-233b-4a9d-86ad-a913140f3430'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:42.339202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-b12584d2-233b-4a9d-86ad-a913140f3430 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64ec77769e724c6b8f54f8c64e3f6dca', 'request_id': 'req-b12584d2-233b-4a9d-86ad-a913140f3430'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:42.386405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-b12584d2-233b-4a9d-86ad-a913140f3430 admin admin] The token cache is being invalidate because project 64ec77769e724c6b8f54f8c64e3f6dca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:02:42.387019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-b12584d2-233b-4a9d-86ad-a913140f3430 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:42.388015 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-b12584d2-233b-4a9d-86ad-a913140f3430 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:42.399747 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b242939c-f7ff-475c-9bf3-5ba0f6faa109 service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:42.400427 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 824/1670] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:02:42 2022] DELETE /identity/v3/projects/64ec77769e724c6b8f54f8c64e3f6dca => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:42.412066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b242939c-f7ff-475c-9bf3-5ba0f6faa109 service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:42.583998 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44725f04-005f-477c-b200-6dd3b6a645da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:42.584606 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44725f04-005f-477c-b200-6dd3b6a645da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:42.584606 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44725f04-005f-477c-b200-6dd3b6a645da admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:42.584742 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44725f04-005f-477c-b200-6dd3b6a645da admin admin] PATH_INFO: `/v3/users/564c041d8a0e4d9990f9010a4d8db400` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:42.587126 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44725f04-005f-477c-b200-6dd3b6a645da admin admin] RBAC: Authorizing `identity:delete_user(user_id=564c041d8a0e4d9990f9010a4d8db400)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:42.595915 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44725f04-005f-477c-b200-6dd3b6a645da admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:42.662346 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12584d2-233b-4a9d-86ad-a913140f3430 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:42.665147 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-b12584d2-233b-4a9d-86ad-a913140f3430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:42.693994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12584d2-233b-4a9d-86ad-a913140f3430 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:42.702969 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3101c62c-f838-4651-84b1-81e16b4ebb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:42.704822 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101c62c-f838-4651-84b1-81e16b4ebb0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:42.705425 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101c62c-f838-4651-84b1-81e16b4ebb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:42.706180 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101c62c-f838-4651-84b1-81e16b4ebb0e admin admin] PATH_INFO: `/v3/users/2243e27d5cec4ea7a5376d438a83f745` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:42.707798 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3101c62c-f838-4651-84b1-81e16b4ebb0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2243e27d5cec4ea7a5376d438a83f745)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:42.715022 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3101c62c-f838-4651-84b1-81e16b4ebb0e admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:42.751100 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-44725f04-005f-477c-b200-6dd3b6a645da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '564c041d8a0e4d9990f9010a4d8db400'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:42.771404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-44725f04-005f-477c-b200-6dd3b6a645da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '564c041d8a0e4d9990f9010a4d8db400'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:42.782129 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-44725f04-005f-477c-b200-6dd3b6a645da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '564c041d8a0e4d9990f9010a4d8db400'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:42.820028 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-44725f04-005f-477c-b200-6dd3b6a645da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '564c041d8a0e4d9990f9010a4d8db400'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:42.822496 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-44725f04-005f-477c-b200-6dd3b6a645da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '564c041d8a0e4d9990f9010a4d8db400'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:42.834055 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-3101c62c-f838-4651-84b1-81e16b4ebb0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2243e27d5cec4ea7a5376d438a83f745'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:42.843108 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 847/1671] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:02:42 2022] DELETE /identity/v3/users/564c041d8a0e4d9990f9010a4d8db400 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:42.854318 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-3101c62c-f838-4651-84b1-81e16b4ebb0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2243e27d5cec4ea7a5376d438a83f745'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:42.866862 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:INSERT=3,keystone:SELECT=81,keystone:DELETE=16 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:42.878118 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-3101c62c-f838-4651-84b1-81e16b4ebb0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2243e27d5cec4ea7a5376d438a83f745'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:42.879954 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-3101c62c-f838-4651-84b1-81e16b4ebb0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2243e27d5cec4ea7a5376d438a83f745'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:42.892371 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-3101c62c-f838-4651-84b1-81e16b4ebb0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2243e27d5cec4ea7a5376d438a83f745'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:42.907225 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 825/1672] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:02:42 2022] DELETE /identity/v3/users/2243e27d5cec4ea7a5376d438a83f745 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:02:42.916835 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0abe5-9366-4d5d-8d76-7f763da9afc3 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:42.917288 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0abe5-9366-4d5d-8d76-7f763da9afc3 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:42.917722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0abe5-9366-4d5d-8d76-7f763da9afc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:42.940778 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-1cb0abe5-9366-4d5d-8d76-7f763da9afc3 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:42.942118 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-1cb0abe5-9366-4d5d-8d76-7f763da9afc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:42.951770 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:43.020836 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 848/1673] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:43.042178 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3101c62c-f838-4651-84b1-81e16b4ebb0e admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:43.043823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-3101c62c-f838-4651-84b1-81e16b4ebb0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:43.216157 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3101c62c-f838-4651-84b1-81e16b4ebb0e admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:43.228998 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e90952-5b02-4fe4-a18c-62e3104e8fc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:43.230164 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e90952-5b02-4fe4-a18c-62e3104e8fc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:43.230833 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e90952-5b02-4fe4-a18c-62e3104e8fc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:43.231390 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e90952-5b02-4fe4-a18c-62e3104e8fc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:43.259872 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e90952-5b02-4fe4-a18c-62e3104e8fc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:43.261348 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e90952-5b02-4fe4-a18c-62e3104e8fc0 service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:43.274203 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 826/1674] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:45.110432 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0abe5-9366-4d5d-8d76-7f763da9afc3 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:45.113061 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-1cb0abe5-9366-4d5d-8d76-7f763da9afc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:45.139755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0abe5-9366-4d5d-8d76-7f763da9afc3 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:45.152678 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5051c1-8541-4f1f-9d49-a80b56dca8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:45.153540 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5051c1-8541-4f1f-9d49-a80b56dca8f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:45.155551 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5051c1-8541-4f1f-9d49-a80b56dca8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:45.155938 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5051c1-8541-4f1f-9d49-a80b56dca8f0 admin admin] PATH_INFO: `/v3/projects/a78bb9942fcc4ef896ee6f8eee6b4935` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:45.157502 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5051c1-8541-4f1f-9d49-a80b56dca8f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a78bb9942fcc4ef896ee6f8eee6b4935)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:45.165062 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5051c1-8541-4f1f-9d49-a80b56dca8f0 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:45.261897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-8e5051c1-8541-4f1f-9d49-a80b56dca8f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a78bb9942fcc4ef896ee6f8eee6b4935', 'request_id': 'req-8e5051c1-8541-4f1f-9d49-a80b56dca8f0'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:45.271514 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-8e5051c1-8541-4f1f-9d49-a80b56dca8f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a78bb9942fcc4ef896ee6f8eee6b4935', 'request_id': 'req-8e5051c1-8541-4f1f-9d49-a80b56dca8f0'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:45.287923 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e90952-5b02-4fe4-a18c-62e3104e8fc0 service neutron] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:45.289516 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-64e90952-5b02-4fe4-a18c-62e3104e8fc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:45.295755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-8e5051c1-8541-4f1f-9d49-a80b56dca8f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a78bb9942fcc4ef896ee6f8eee6b4935', 'request_id': 'req-8e5051c1-8541-4f1f-9d49-a80b56dca8f0'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:45.307298 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e90952-5b02-4fe4-a18c-62e3104e8fc0 service neutron] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:45.324112 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-8e5051c1-8541-4f1f-9d49-a80b56dca8f0 admin admin] The token cache is being invalidate because project a78bb9942fcc4ef896ee6f8eee6b4935 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:02:45.326431 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-8e5051c1-8541-4f1f-9d49-a80b56dca8f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:45.328482 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-8e5051c1-8541-4f1f-9d49-a80b56dca8f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:45.330204 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 849/1675] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:02:45 2022] DELETE /identity/v3/projects/a78bb9942fcc4ef896ee6f8eee6b4935 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:45.335325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5051c1-8541-4f1f-9d49-a80b56dca8f0 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:45.338580 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-8e5051c1-8541-4f1f-9d49-a80b56dca8f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:45.446456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238d6a57-71c3-46be-a72e-36613c6b7997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:45.447426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d6a57-71c3-46be-a72e-36613c6b7997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:45.448293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d6a57-71c3-46be-a72e-36613c6b7997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:45.448766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d6a57-71c3-46be-a72e-36613c6b7997 admin admin] PATH_INFO: `/v3/projects/dd20569bd4c1444f81cfb700f5d1e20a` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:45.452328 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238d6a57-71c3-46be-a72e-36613c6b7997 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd20569bd4c1444f81cfb700f5d1e20a)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:45.454440 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238d6a57-71c3-46be-a72e-36613c6b7997 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:45.508703 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5051c1-8541-4f1f-9d49-a80b56dca8f0 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:45.513624 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6bef3b-c500-44fb-ae6d-727acf5ab7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:45.516816 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6bef3b-c500-44fb-ae6d-727acf5ab7a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:45.517183 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6bef3b-c500-44fb-ae6d-727acf5ab7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:45.517541 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6bef3b-c500-44fb-ae6d-727acf5ab7a5 admin admin] PATH_INFO: `/v3/users/effe6d1e7d8e47a5a5921f683c0b9444` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:45.518894 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6bef3b-c500-44fb-ae6d-727acf5ab7a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=effe6d1e7d8e47a5a5921f683c0b9444)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:45.521743 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6bef3b-c500-44fb-ae6d-727acf5ab7a5 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:45.597303 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-238d6a57-71c3-46be-a72e-36613c6b7997 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd20569bd4c1444f81cfb700f5d1e20a', 'request_id': 'req-238d6a57-71c3-46be-a72e-36613c6b7997'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:45.611257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-238d6a57-71c3-46be-a72e-36613c6b7997 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd20569bd4c1444f81cfb700f5d1e20a', 'request_id': 'req-238d6a57-71c3-46be-a72e-36613c6b7997'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:45.614706 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:45.627005 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-238d6a57-71c3-46be-a72e-36613c6b7997 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd20569bd4c1444f81cfb700f5d1e20a', 'request_id': 'req-238d6a57-71c3-46be-a72e-36613c6b7997'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:45.634630 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ad6bef3b-c500-44fb-ae6d-727acf5ab7a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'effe6d1e7d8e47a5a5921f683c0b9444'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:45.656588 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-238d6a57-71c3-46be-a72e-36613c6b7997 admin admin] The token cache is being invalidate because project dd20569bd4c1444f81cfb700f5d1e20a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:02:45.656889 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-238d6a57-71c3-46be-a72e-36613c6b7997 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:45.658948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ad6bef3b-c500-44fb-ae6d-727acf5ab7a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'effe6d1e7d8e47a5a5921f683c0b9444'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:45.659431 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-238d6a57-71c3-46be-a72e-36613c6b7997 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:45.662932 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 827/1676] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:02:45 2022] DELETE /identity/v3/projects/dd20569bd4c1444f81cfb700f5d1e20a => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:45.667833 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815bf0d-e5db-4b04-af2d-964320087887 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:45.668338 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815bf0d-e5db-4b04-af2d-964320087887 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:45.668819 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815bf0d-e5db-4b04-af2d-964320087887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:45.674503 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ad6bef3b-c500-44fb-ae6d-727acf5ab7a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'effe6d1e7d8e47a5a5921f683c0b9444'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:45.711823 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ad6bef3b-c500-44fb-ae6d-727acf5ab7a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'effe6d1e7d8e47a5a5921f683c0b9444'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:45.712862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ad6bef3b-c500-44fb-ae6d-727acf5ab7a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'effe6d1e7d8e47a5a5921f683c0b9444'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:45.714752 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-6815bf0d-e5db-4b04-af2d-964320087887 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:45.723761 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-6815bf0d-e5db-4b04-af2d-964320087887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:45.728753 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 850/1677] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:02:45 2022] DELETE /identity/v3/users/effe6d1e7d8e47a5a5921f683c0b9444 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:45.734220 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:45.841075 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 828/1678] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:45.842291 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d028e8-95e6-481f-8288-caedde3f3ac9 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:45.842597 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d028e8-95e6-481f-8288-caedde3f3ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:45.842868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d028e8-95e6-481f-8288-caedde3f3ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:45.873633 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a797d1-fd25-47d2-999c-f3e82720a32e None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:45.873927 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a797d1-fd25-47d2-999c-f3e82720a32e None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:45.874208 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a797d1-fd25-47d2-999c-f3e82720a32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:45.881891 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-a5d028e8-95e6-481f-8288-caedde3f3ac9 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:45.883123 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-a5d028e8-95e6-481f-8288-caedde3f3ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:45.906340 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 851/1679] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:45.906340 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d028e8-95e6-481f-8288-caedde3f3ac9 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:45.935727 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d028e8-95e6-481f-8288-caedde3f3ac9 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:45.940464 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-91a797d1-fd25-47d2-999c-f3e82720a32e None None] MFA Rules not processed for user `13a8b01944e241bfb9fd9c2e30c2fc43`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:45.940464 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-91a797d1-fd25-47d2-999c-f3e82720a32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:45.952412 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea052aa-c6e2-423a-9776-e34ca8a3006b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:45.953374 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea052aa-c6e2-423a-9776-e34ca8a3006b admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:45.953792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea052aa-c6e2-423a-9776-e34ca8a3006b admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:45.954355 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea052aa-c6e2-423a-9776-e34ca8a3006b admin admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:45.955500 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea052aa-c6e2-423a-9776-e34ca8a3006b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:02:45.955952 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea052aa-c6e2-423a-9776-e34ca8a3006b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:45.959564 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea052aa-c6e2-423a-9776-e34ca8a3006b admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:45.988688 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 852/1680] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 20:02:45.991118 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea052aa-c6e2-423a-9776-e34ca8a3006b admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:46.014015 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea052aa-c6e2-423a-9776-e34ca8a3006b admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:46.024295 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e462a6bd-ef1c-4f94-b59b-35aee9bc078d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:46.024806 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e462a6bd-ef1c-4f94-b59b-35aee9bc078d admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:46.025152 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e462a6bd-ef1c-4f94-b59b-35aee9bc078d admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:46.025512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e462a6bd-ef1c-4f94-b59b-35aee9bc078d admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:46.027338 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e462a6bd-ef1c-4f94-b59b-35aee9bc078d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:46.043659 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e462a6bd-ef1c-4f94-b59b-35aee9bc078d admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:46.096774 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 853/1681] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:02:45 2022] POST /identity/v3/projects => generated 363 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:46.099235 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e462a6bd-ef1c-4f94-b59b-35aee9bc078d admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:46.108531 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 829/1682] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:45 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:46.113037 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a797d1-fd25-47d2-999c-f3e82720a32e None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:46.117428 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-91a797d1-fd25-47d2-999c-f3e82720a32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:46.134442 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a797d1-fd25-47d2-999c-f3e82720a32e None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:46.205000 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8442c91-d4c4-410a-9b83-6e88b72e80b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:46.205636 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8442c91-d4c4-410a-9b83-6e88b72e80b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:46.205921 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8442c91-d4c4-410a-9b83-6e88b72e80b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:46.206355 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8442c91-d4c4-410a-9b83-6e88b72e80b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:46.207041 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8442c91-d4c4-410a-9b83-6e88b72e80b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:46.209405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8442c91-d4c4-410a-9b83-6e88b72e80b6 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:46.261600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e462a6bd-ef1c-4f94-b59b-35aee9bc078d admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:46.275415 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be76149-def4-4209-931a-00c16110b586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:46.279197 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be76149-def4-4209-931a-00c16110b586 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:46.279540 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be76149-def4-4209-931a-00c16110b586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:46.279970 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be76149-def4-4209-931a-00c16110b586 admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:46.281270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be76149-def4-4209-931a-00c16110b586 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:46.286820 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be76149-def4-4209-931a-00c16110b586 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:46.290912 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 830/1683] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:02:46 2022] POST /identity/v3/projects => generated 403 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:46.295109 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8442c91-d4c4-410a-9b83-6e88b72e80b6 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:46.360703 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 854/1684] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:02:46 2022] POST /identity/v3/users => generated 282 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:02:46.365801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be76149-def4-4209-931a-00c16110b586 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:46.403684 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be76149-def4-4209-931a-00c16110b586 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:46.430719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:46.601327 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8442c91-d4c4-410a-9b83-6e88b72e80b6 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:46.623689 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c65ae-adcc-4918-b7d8-684440b3f38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:46.624485 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c65ae-adcc-4918-b7d8-684440b3f38e admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:46.625895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c65ae-adcc-4918-b7d8-684440b3f38e admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:46.626419 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c65ae-adcc-4918-b7d8-684440b3f38e admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:46.628918 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430c65ae-adcc-4918-b7d8-684440b3f38e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:46.632092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430c65ae-adcc-4918-b7d8-684440b3f38e admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:46.669841 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c611cb-85fc-4797-afa1-b89328e4b56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:46.670502 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c611cb-85fc-4797-afa1-b89328e4b56a admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:46.670788 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c611cb-85fc-4797-afa1-b89328e4b56a admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:46.681019 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c611cb-85fc-4797-afa1-b89328e4b56a admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:46.683545 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c611cb-85fc-4797-afa1-b89328e4b56a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:46.686850 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c611cb-85fc-4797-afa1-b89328e4b56a admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:46.700278 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:46.724863 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 831/1685] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:02:46 2022] POST /identity/v3/users => generated 308 bytes in 430 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:46.725932 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c65ae-adcc-4918-b7d8-684440b3f38e admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:46.747614 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c65ae-adcc-4918-b7d8-684440b3f38e admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:46.749017 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 855/1686] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:46.753405 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c611cb-85fc-4797-afa1-b89328e4b56a admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:46.772104 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c611cb-85fc-4797-afa1-b89328e4b56a admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:46.774543 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc13a5e4-24af-40d7-9a16-7f1110c52613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:46.775184 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13a5e4-24af-40d7-9a16-7f1110c52613 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:46.775464 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13a5e4-24af-40d7-9a16-7f1110c52613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:46.775731 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13a5e4-24af-40d7-9a16-7f1110c52613 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:46.776440 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc13a5e4-24af-40d7-9a16-7f1110c52613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:46.778188 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc13a5e4-24af-40d7-9a16-7f1110c52613 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:46.786449 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772e9c7-0174-4ed2-828b-322652af95ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:46.787156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772e9c7-0174-4ed2-828b-322652af95ed admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:46.787445 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772e9c7-0174-4ed2-828b-322652af95ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:46.787727 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772e9c7-0174-4ed2-828b-322652af95ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:46.788630 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a772e9c7-0174-4ed2-828b-322652af95ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:46.792435 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a772e9c7-0174-4ed2-828b-322652af95ed admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:46.797439 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 832/1687] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:46.801884 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc13a5e4-24af-40d7-9a16-7f1110c52613 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:46.822537 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc13a5e4-24af-40d7-9a16-7f1110c52613 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:46.823248 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 856/1688] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:02:46.824164 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772e9c7-0174-4ed2-828b-322652af95ed admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:46.836906 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b06d9-0cb0-4288-8579-2f3fe3e4f0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:46.837630 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b06d9-0cb0-4288-8579-2f3fe3e4f0ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:46.837914 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b06d9-0cb0-4288-8579-2f3fe3e4f0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:46.838189 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b06d9-0cb0-4288-8579-2f3fe3e4f0ff admin admin] PATH_INFO: `/v3/projects/f4c29967ff11466f8f8780d067b24824/users/3569fb5aae9b4397b573020ab6bd2091/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:46.840320 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772e9c7-0174-4ed2-828b-322652af95ed admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:46.848641 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9c035-fa8e-4701-b4cf-c3ef1d84c788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:46.849943 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9c035-fa8e-4701-b4cf-c3ef1d84c788 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:46.850666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9c035-fa8e-4701-b4cf-c3ef1d84c788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:46.851843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9c035-fa8e-4701-b4cf-c3ef1d84c788 admin admin] PATH_INFO: `/v3/projects/792c790017474d1399dfe323d2a8a299/users/c79ba4a88bf04becbd97c8f83748d475/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:46.870895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9b06d9-0cb0-4288-8579-2f3fe3e4f0ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4c29967ff11466f8f8780d067b24824, user_id=3569fb5aae9b4397b573020ab6bd2091, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:46.873643 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9b06d9-0cb0-4288-8579-2f3fe3e4f0ff admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:46.878786 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9c035-fa8e-4701-b4cf-c3ef1d84c788 admin admin] RBAC: Authorizing `identity:create_grant(project_id=792c790017474d1399dfe323d2a8a299, user_id=c79ba4a88bf04becbd97c8f83748d475, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:46.883260 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9c035-fa8e-4701-b4cf-c3ef1d84c788 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:46.912220 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 857/1689] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:02:46 2022] PUT /identity/v3/projects/792c790017474d1399dfe323d2a8a299/users/c79ba4a88bf04becbd97c8f83748d475/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:46.915215 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 833/1690] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:02:46 2022] PUT /identity/v3/projects/f4c29967ff11466f8f8780d067b24824/users/3569fb5aae9b4397b573020ab6bd2091/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:02:46.921427 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9c035-fa8e-4701-b4cf-c3ef1d84c788 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:46.927814 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42de4e-c9c8-4e5a-a716-ed18dbbe018c None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:46.929272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42de4e-c9c8-4e5a-a716-ed18dbbe018c None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:46.931445 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42de4e-c9c8-4e5a-a716-ed18dbbe018c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:47.009557 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-5d42de4e-c9c8-4e5a-a716-ed18dbbe018c None None] MFA Rules not processed for user `c79ba4a88bf04becbd97c8f83748d475`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:47.011612 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-5d42de4e-c9c8-4e5a-a716-ed18dbbe018c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:47.080396 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9c035-fa8e-4701-b4cf-c3ef1d84c788 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:47.186518 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b197f-2568-476f-bc06-7c7e097a7fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:47.189041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b197f-2568-476f-bc06-7c7e097a7fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:47.190309 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b197f-2568-476f-bc06-7c7e097a7fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:47.190864 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b197f-2568-476f-bc06-7c7e097a7fa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:47.191609 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4b197f-2568-476f-bc06-7c7e097a7fa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:47.192652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 834/1691] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:47.197081 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4b197f-2568-476f-bc06-7c7e097a7fa5 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:47.198024 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6503a7-3bf1-487b-a4fc-035ece0780a2 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:47.204986 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6503a7-3bf1-487b-a4fc-035ece0780a2 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:47.205309 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6503a7-3bf1-487b-a4fc-035ece0780a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:47.212805 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 858/1692] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:47.213786 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b197f-2568-476f-bc06-7c7e097a7fa5 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:47.228438 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b197f-2568-476f-bc06-7c7e097a7fa5 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:47.233906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b714a7-4b75-411b-a1e2-0f80f7f68356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:47.234621 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b714a7-4b75-411b-a1e2-0f80f7f68356 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:47.234932 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b714a7-4b75-411b-a1e2-0f80f7f68356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:47.235211 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b714a7-4b75-411b-a1e2-0f80f7f68356 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:47.235879 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b714a7-4b75-411b-a1e2-0f80f7f68356 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:47.238522 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b714a7-4b75-411b-a1e2-0f80f7f68356 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:47.287772 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-ce6503a7-3bf1-487b-a4fc-035ece0780a2 None None] MFA Rules not processed for user `293b8a62f19c45c1bf5b897c7cf40edb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:47.289359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-ce6503a7-3bf1-487b-a4fc-035ece0780a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:47.306875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 859/1693] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:02:47 2022] POST /identity/v3/projects => generated 365 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:47.306875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b714a7-4b75-411b-a1e2-0f80f7f68356 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:47.412289 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:47.427097 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 835/1694] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:47 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:47.428898 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:47.429620 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6503a7-3bf1-487b-a4fc-035ece0780a2 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:47.448970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6503a7-3bf1-487b-a4fc-035ece0780a2 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:47.461047 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a8231-c3bb-4238-9df6-1cf4c2d2da32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:47.462072 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a8231-c3bb-4238-9df6-1cf4c2d2da32 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:47.462437 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a8231-c3bb-4238-9df6-1cf4c2d2da32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:47.462785 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a8231-c3bb-4238-9df6-1cf4c2d2da32 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:47.463581 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a8231-c3bb-4238-9df6-1cf4c2d2da32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:47.465181 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b714a7-4b75-411b-a1e2-0f80f7f68356 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:47.465899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a8231-c3bb-4238-9df6-1cf4c2d2da32 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:47.474572 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d7aac-9101-4e27-b922-673f4c3e8471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:47.475896 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d7aac-9101-4e27-b922-673f4c3e8471 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:47.476636 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d7aac-9101-4e27-b922-673f4c3e8471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:47.477179 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d7aac-9101-4e27-b922-673f4c3e8471 admin admin] PATH_INFO: `/v3/projects/f4c29967ff11466f8f8780d067b24824/users/3569fb5aae9b4397b573020ab6bd2091/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:47.485755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d7aac-9101-4e27-b922-673f4c3e8471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4c29967ff11466f8f8780d067b24824, user_id=3569fb5aae9b4397b573020ab6bd2091, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:47.489944 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d7aac-9101-4e27-b922-673f4c3e8471 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:47.508679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 836/1695] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:02:47 2022] POST /identity/v3/users => generated 289 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:47.508968 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 860/1696] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:02:47 2022] PUT /identity/v3/projects/f4c29967ff11466f8f8780d067b24824/users/3569fb5aae9b4397b573020ab6bd2091/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:47.509046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a8231-c3bb-4238-9df6-1cf4c2d2da32 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:47.509117 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dba2cb-1a8e-4b2f-a342-e8a2b5303899 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:47.509117 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dba2cb-1a8e-4b2f-a342-e8a2b5303899 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:47.509245 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-901a8231-c3bb-4238-9df6-1cf4c2d2da32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:47.509312 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dba2cb-1a8e-4b2f-a342-e8a2b5303899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:47.546767 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-60dba2cb-1a8e-4b2f-a342-e8a2b5303899 None None] MFA Rules not processed for user `1afe13c8fbb1468b8ef3d0d0c671208d`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:47.548114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-60dba2cb-1a8e-4b2f-a342-e8a2b5303899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:47.557741 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a8231-c3bb-4238-9df6-1cf4c2d2da32 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:47.719102 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 861/1697] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:47.720980 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8826738-4ed1-4e92-bbdc-0eefa488544d None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:47.720980 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8826738-4ed1-4e92-bbdc-0eefa488544d None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:47.721342 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8826738-4ed1-4e92-bbdc-0eefa488544d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:47.749581 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad18a768-89fb-48a2-96a3-84d1b03e944c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:47.750649 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18a768-89fb-48a2-96a3-84d1b03e944c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:47.750946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18a768-89fb-48a2-96a3-84d1b03e944c admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:47.751226 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18a768-89fb-48a2-96a3-84d1b03e944c admin admin] PATH_INFO: `/v3/projects/32a5ef819ffd4760acd1a9cc88afe165` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:47.753540 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-f8826738-4ed1-4e92-bbdc-0eefa488544d None None] MFA Rules not processed for user `293b8a62f19c45c1bf5b897c7cf40edb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:47.756364 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-f8826738-4ed1-4e92-bbdc-0eefa488544d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:47.758877 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad18a768-89fb-48a2-96a3-84d1b03e944c admin admin] RBAC: Authorizing `identity:delete_project(project_id=32a5ef819ffd4760acd1a9cc88afe165)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:47.760787 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad18a768-89fb-48a2-96a3-84d1b03e944c admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:47.854152 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 862/1698] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:47 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:47.867142 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8826738-4ed1-4e92-bbdc-0eefa488544d None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:47.902745 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8826738-4ed1-4e92-bbdc-0eefa488544d None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:47.920351 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ad18a768-89fb-48a2-96a3-84d1b03e944c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32a5ef819ffd4760acd1a9cc88afe165', 'request_id': 'req-ad18a768-89fb-48a2-96a3-84d1b03e944c'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:47.938768 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ad18a768-89fb-48a2-96a3-84d1b03e944c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32a5ef819ffd4760acd1a9cc88afe165', 'request_id': 'req-ad18a768-89fb-48a2-96a3-84d1b03e944c'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:47.949443 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ad18a768-89fb-48a2-96a3-84d1b03e944c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32a5ef819ffd4760acd1a9cc88afe165', 'request_id': 'req-ad18a768-89fb-48a2-96a3-84d1b03e944c'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:47.979070 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ad18a768-89fb-48a2-96a3-84d1b03e944c admin admin] The token cache is being invalidate because project 32a5ef819ffd4760acd1a9cc88afe165 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:02:47.981337 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ad18a768-89fb-48a2-96a3-84d1b03e944c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:47.983481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ad18a768-89fb-48a2-96a3-84d1b03e944c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:47.991034 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 837/1699] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:02:47 2022] DELETE /identity/v3/projects/32a5ef819ffd4760acd1a9cc88afe165 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:47.991034 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad18a768-89fb-48a2-96a3-84d1b03e944c admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:47.991768 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-ad18a768-89fb-48a2-96a3-84d1b03e944c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:48.056173 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a63dd0-1247-4b1f-b459-0b6a70802ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:48.057014 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a63dd0-1247-4b1f-b459-0b6a70802ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:48.057435 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a63dd0-1247-4b1f-b459-0b6a70802ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:48.057772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a63dd0-1247-4b1f-b459-0b6a70802ce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:48.058531 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a63dd0-1247-4b1f-b459-0b6a70802ce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:48.060253 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a63dd0-1247-4b1f-b459-0b6a70802ce5 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:48.103589 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 863/1700] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:48.103589 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a63dd0-1247-4b1f-b459-0b6a70802ce5 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:48.113260 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-c8a63dd0-1247-4b1f-b459-0b6a70802ce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:48.162318 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:48.249567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad18a768-89fb-48a2-96a3-84d1b03e944c admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:48.270695 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ad282-90ac-487b-8ceb-909046f5c6ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:48.271581 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ad282-90ac-487b-8ceb-909046f5c6ba service nova] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:48.271842 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ad282-90ac-487b-8ceb-909046f5c6ba service nova] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:48.272748 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ad282-90ac-487b-8ceb-909046f5c6ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:48.282945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-872ad282-90ac-487b-8ceb-909046f5c6ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:48.303458 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:48.369423 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a63dd0-1247-4b1f-b459-0b6a70802ce5 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:48.386585 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1436461-5f57-4e42-8e6f-b3bf14573680 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:48.387269 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1436461-5f57-4e42-8e6f-b3bf14573680 service nova] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:48.387552 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1436461-5f57-4e42-8e6f-b3bf14573680 service nova] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:48.387829 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1436461-5f57-4e42-8e6f-b3bf14573680 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:48.391947 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-b1436461-5f57-4e42-8e6f-b3bf14573680 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:48.437760 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ad282-90ac-487b-8ceb-909046f5c6ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:48.445565 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ad282-90ac-487b-8ceb-909046f5c6ba service nova] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:48.465788 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 838/1701] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:48.467174 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f6f47-9a0e-4b25-a420-29b192a58f43 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:48.467447 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f6f47-9a0e-4b25-a420-29b192a58f43 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:48.467720 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f6f47-9a0e-4b25-a420-29b192a58f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:48.540521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-3d9f6f47-9a0e-4b25-a420-29b192a58f43 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:48.545962 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-3d9f6f47-9a0e-4b25-a420-29b192a58f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:48.564167 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1436461-5f57-4e42-8e6f-b3bf14573680 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:48.566234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1436461-5f57-4e42-8e6f-b3bf14573680 service nova] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:48.581394 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 839/1702] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:48.582287 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f6f47-9a0e-4b25-a420-29b192a58f43 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:48.591577 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-3d9f6f47-9a0e-4b25-a420-29b192a58f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:48.602731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 864/1703] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:48 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:48.637081 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5beb4-80fc-430d-aa14-558ab04ebe6b None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:48.637492 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5beb4-80fc-430d-aa14-558ab04ebe6b None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:48.637954 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5beb4-80fc-430d-aa14-558ab04ebe6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:48.644499 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f6f47-9a0e-4b25-a420-29b192a58f43 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:48.677053 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d659ba0-0435-4588-aee7-e9c47b6993e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:48.679730 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d659ba0-0435-4588-aee7-e9c47b6993e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:48.680016 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d659ba0-0435-4588-aee7-e9c47b6993e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:48.680293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d659ba0-0435-4588-aee7-e9c47b6993e8 admin admin] PATH_INFO: `/v3/projects/f1d08717b2e340dd8a1bb6d56310ae53/users/9762ad5809074941a3257f148d01e560/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:48.689775 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-7df5beb4-80fc-430d-aa14-558ab04ebe6b None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:48.693103 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-7df5beb4-80fc-430d-aa14-558ab04ebe6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:48.725842 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 865/1704] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:48.731594 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5beb4-80fc-430d-aa14-558ab04ebe6b None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:48.759283 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d659ba0-0435-4588-aee7-e9c47b6993e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1d08717b2e340dd8a1bb6d56310ae53, user_id=9762ad5809074941a3257f148d01e560, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:48.765753 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d659ba0-0435-4588-aee7-e9c47b6993e8 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:48.767770 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5beb4-80fc-430d-aa14-558ab04ebe6b None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:48.790593 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b187e7-7f50-40e6-8d52-a81929badcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:48.794219 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b187e7-7f50-40e6-8d52-a81929badcef admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:48.795229 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b187e7-7f50-40e6-8d52-a81929badcef admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:48.795533 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b187e7-7f50-40e6-8d52-a81929badcef admin admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:48.797120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b187e7-7f50-40e6-8d52-a81929badcef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:02:48.802121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 840/1705] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:02:48 2022] PUT /identity/v3/projects/f1d08717b2e340dd8a1bb6d56310ae53/users/9762ad5809074941a3257f148d01e560/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:48.803500 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d659ba0-0435-4588-aee7-e9c47b6993e8 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:48.803838 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b187e7-7f50-40e6-8d52-a81929badcef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:48.810971 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b187e7-7f50-40e6-8d52-a81929badcef admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:48.856979 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 866/1706] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:02:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:02:48.863082 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b187e7-7f50-40e6-8d52-a81929badcef admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:48.907343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b187e7-7f50-40e6-8d52-a81929badcef admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:48.964658 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d659ba0-0435-4588-aee7-e9c47b6993e8 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:48.977427 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafbc43b-0ecd-49f8-8f79-f9d361c70412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:48.979247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbc43b-0ecd-49f8-8f79-f9d361c70412 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:48.979247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbc43b-0ecd-49f8-8f79-f9d361c70412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:48.979247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbc43b-0ecd-49f8-8f79-f9d361c70412 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:48.979578 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafbc43b-0ecd-49f8-8f79-f9d361c70412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:48.981033 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafbc43b-0ecd-49f8-8f79-f9d361c70412 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:49.006606 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 841/1707] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:49.021449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafbc43b-0ecd-49f8-8f79-f9d361c70412 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:49.042243 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafbc43b-0ecd-49f8-8f79-f9d361c70412 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:49.052766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1afd63-4fc1-4188-a584-0f278a9d34a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:49.054594 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1afd63-4fc1-4188-a584-0f278a9d34a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:49.054898 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1afd63-4fc1-4188-a584-0f278a9d34a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:49.055444 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1afd63-4fc1-4188-a584-0f278a9d34a9 admin admin] PATH_INFO: `/v3/projects/f1d08717b2e340dd8a1bb6d56310ae53/users/9762ad5809074941a3257f148d01e560/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:49.061521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1afd63-4fc1-4188-a584-0f278a9d34a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1d08717b2e340dd8a1bb6d56310ae53, user_id=9762ad5809074941a3257f148d01e560, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:49.064184 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fefb6-2f29-4f5d-bc66-b443d5dddbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:49.064904 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fefb6-2f29-4f5d-bc66-b443d5dddbcc admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:49.065263 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fefb6-2f29-4f5d-bc66-b443d5dddbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:49.065609 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fefb6-2f29-4f5d-bc66-b443d5dddbcc admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:49.066405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1afd63-4fc1-4188-a584-0f278a9d34a9 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:49.066555 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207fefb6-2f29-4f5d-bc66-b443d5dddbcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:49.070007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207fefb6-2f29-4f5d-bc66-b443d5dddbcc admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:49.097684 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 842/1708] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:02:49 2022] PUT /identity/v3/projects/f1d08717b2e340dd8a1bb6d56310ae53/users/9762ad5809074941a3257f148d01e560/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:49.108801 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75180c-53dd-4579-bef0-bb615814f607 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:49.109038 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75180c-53dd-4579-bef0-bb615814f607 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:49.109312 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75180c-53dd-4579-bef0-bb615814f607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:49.159766 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 867/1709] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:02:48 2022] POST /identity/v3/projects => generated 381 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:49.164129 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fefb6-2f29-4f5d-bc66-b443d5dddbcc admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:49.189314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:49.202144 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-fa75180c-53dd-4579-bef0-bb615814f607 None None] MFA Rules not processed for user `9762ad5809074941a3257f148d01e560`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:49.203381 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-fa75180c-53dd-4579-bef0-bb615814f607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:49.272520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fefb6-2f29-4f5d-bc66-b443d5dddbcc admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:49.278817 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28eccd5-5d88-49d0-a0cc-36e95e294daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:49.280116 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28eccd5-5d88-49d0-a0cc-36e95e294daf admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:49.280403 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28eccd5-5d88-49d0-a0cc-36e95e294daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:49.280728 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28eccd5-5d88-49d0-a0cc-36e95e294daf admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:49.281859 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28eccd5-5d88-49d0-a0cc-36e95e294daf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:49.287120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28eccd5-5d88-49d0-a0cc-36e95e294daf admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:49.325240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 868/1710] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:02:49 2022] POST /identity/v3/users => generated 291 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:02:49.326377 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28eccd5-5d88-49d0-a0cc-36e95e294daf admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:49.331220 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:49.343049 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28eccd5-5d88-49d0-a0cc-36e95e294daf admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:49.356944 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d2e201-6326-4c78-8f54-1534e98f15fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:49.357578 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2e201-6326-4c78-8f54-1534e98f15fc admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:49.358051 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2e201-6326-4c78-8f54-1534e98f15fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:49.358330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2e201-6326-4c78-8f54-1534e98f15fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:49.359096 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d2e201-6326-4c78-8f54-1534e98f15fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:49.360679 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d2e201-6326-4c78-8f54-1534e98f15fc admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:49.379748 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 843/1711] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:49 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:02:49.381001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 869/1712] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:02:49.387495 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa75180c-53dd-4579-bef0-bb615814f607 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:49.393420 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d2e201-6326-4c78-8f54-1534e98f15fc admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:49.396685 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-f1d2e201-6326-4c78-8f54-1534e98f15fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:49.411184 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa75180c-53dd-4579-bef0-bb615814f607 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:49.421587 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb0cdd-c21e-4656-8443-e383f9596e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:49.422481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb0cdd-c21e-4656-8443-e383f9596e5b admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:49.424470 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb0cdd-c21e-4656-8443-e383f9596e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:49.424856 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb0cdd-c21e-4656-8443-e383f9596e5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:49.425649 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb0cdd-c21e-4656-8443-e383f9596e5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:49.428419 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb0cdd-c21e-4656-8443-e383f9596e5b admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:49.455879 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 844/1713] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:49.458703 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb0cdd-c21e-4656-8443-e383f9596e5b admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:49.475796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb0cdd-c21e-4656-8443-e383f9596e5b admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:49.497330 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733c047-ea8e-4c36-92e6-473400aace1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:49.498091 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733c047-ea8e-4c36-92e6-473400aace1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:49.498657 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733c047-ea8e-4c36-92e6-473400aace1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:49.499414 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733c047-ea8e-4c36-92e6-473400aace1e admin admin] PATH_INFO: `/v3/projects/c86fa13f44f14576983128e4bda934e7/users/7fc6b06f10574871994798ce8673ada4/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:49.511739 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d2e201-6326-4c78-8f54-1534e98f15fc admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:49.523062 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9f8fd-53a6-4446-905e-a38725186357 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:49.523800 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9f8fd-53a6-4446-905e-a38725186357 service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:49.524155 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9f8fd-53a6-4446-905e-a38725186357 service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:49.524500 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9f8fd-53a6-4446-905e-a38725186357 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:49.536311 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2733c047-ea8e-4c36-92e6-473400aace1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c86fa13f44f14576983128e4bda934e7, user_id=7fc6b06f10574871994798ce8673ada4, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:49.539678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2733c047-ea8e-4c36-92e6-473400aace1e admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:49.551793 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9f8fd-53a6-4446-905e-a38725186357 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:49.556412 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9f8fd-53a6-4446-905e-a38725186357 service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:49.569711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 845/1714] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:02:49 2022] PUT /identity/v3/projects/c86fa13f44f14576983128e4bda934e7/users/7fc6b06f10574871994798ce8673ada4/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:49.583699 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384021dc-3613-4a3b-b50e-21c3aad96a6d None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:49.587652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384021dc-3613-4a3b-b50e-21c3aad96a6d None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:49.588015 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384021dc-3613-4a3b-b50e-21c3aad96a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:49.593446 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 870/1715] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:49 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:49.624847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-384021dc-3613-4a3b-b50e-21c3aad96a6d None None] MFA Rules not processed for user `7fc6b06f10574871994798ce8673ada4`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:49.626495 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-384021dc-3613-4a3b-b50e-21c3aad96a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:49.708267 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf5519-5006-4d77-bb01-46255d26d9f3 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:49.708558 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf5519-5006-4d77-bb01-46255d26d9f3 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:49.709059 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf5519-5006-4d77-bb01-46255d26d9f3 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:49.710845 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 871/1716] 10.222.0.39 () {54 vars in 912 bytes} [Fri Aug 5 20:02:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 20:02:49.711970 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7aef8-6f88-4f80-bb8c-5429445c321c None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:49.712244 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7aef8-6f88-4f80-bb8c-5429445c321c None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:49.712520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7aef8-6f88-4f80-bb8c-5429445c321c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:49.817077 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 846/1717] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:49.819211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f636622-a831-43d8-a4ab-c49236a6150d None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:49.819531 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f636622-a831-43d8-a4ab-c49236a6150d None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:49.819814 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f636622-a831-43d8-a4ab-c49236a6150d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:49.826615 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-fde7aef8-6f88-4f80-bb8c-5429445c321c None None] MFA Rules not processed for user `83e975aed96648fc9c06f95d49c6f6f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:49.827922 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-fde7aef8-6f88-4f80-bb8c-5429445c321c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:49.931679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-4f636622-a831-43d8-a4ab-c49236a6150d None None] MFA Rules not processed for user `b1c75368c00e4b0cb7b3163728843479`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:49.933429 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-4f636622-a831-43d8-a4ab-c49236a6150d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:50.060922 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 872/1718] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:50.072811 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde7aef8-6f88-4f80-bb8c-5429445c321c None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:50.126149 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:50.151276 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 847/1719] 10.222.0.39 () {58 vars in 1009 bytes} [Fri Aug 5 20:02:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:50.152555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6d3a3-16f1-4129-8698-7dfff838ba0a None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:50.152869 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6d3a3-16f1-4129-8698-7dfff838ba0a None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:50.153150 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6d3a3-16f1-4129-8698-7dfff838ba0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:50.232267 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-e6d6d3a3-16f1-4129-8698-7dfff838ba0a None None] MFA Rules not processed for user `3569fb5aae9b4397b573020ab6bd2091`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:50.235229 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde7aef8-6f88-4f80-bb8c-5429445c321c None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:50.235977 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-e6d6d3a3-16f1-4129-8698-7dfff838ba0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:50.245015 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b7446-4570-4c31-944c-04377d388659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:50.245714 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b7446-4570-4c31-944c-04377d388659 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:50.246114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b7446-4570-4c31-944c-04377d388659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:50.246402 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b7446-4570-4c31-944c-04377d388659 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:50.247166 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b7446-4570-4c31-944c-04377d388659 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:50.249324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b7446-4570-4c31-944c-04377d388659 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:50.305393 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 873/1720] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:02:50 2022] POST /identity/v3/projects => generated 381 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:50.308882 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9501c-23f3-4ee0-8a9b-565c6570bfea None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:50.309182 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9501c-23f3-4ee0-8a9b-565c6570bfea None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:50.309458 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9501c-23f3-4ee0-8a9b-565c6570bfea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:50.324720 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:50.373307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-c9f9501c-23f3-4ee0-8a9b-565c6570bfea None None] MFA Rules not processed for user `83e975aed96648fc9c06f95d49c6f6f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:50.375210 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-c9f9501c-23f3-4ee0-8a9b-565c6570bfea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:50.445813 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 848/1721] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:50 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:50.459513 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d6d3a3-16f1-4129-8698-7dfff838ba0a None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:50.464195 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-e6d6d3a3-16f1-4129-8698-7dfff838ba0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:50.478689 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d6d3a3-16f1-4129-8698-7dfff838ba0a None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:50.524610 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 874/1722] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:50.525399 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9501c-23f3-4ee0-8a9b-565c6570bfea None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:50.606112 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475848c-5ef7-4812-8229-dd53fdbba68b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:50.607167 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475848c-5ef7-4812-8229-dd53fdbba68b service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:50.609246 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475848c-5ef7-4812-8229-dd53fdbba68b service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:50.609624 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475848c-5ef7-4812-8229-dd53fdbba68b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:50.645280 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9475848c-5ef7-4812-8229-dd53fdbba68b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:50.655686 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9475848c-5ef7-4812-8229-dd53fdbba68b service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:50.686372 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9501c-23f3-4ee0-8a9b-565c6570bfea None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:50.689062 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 849/1723] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:50.692306 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475848c-5ef7-4812-8229-dd53fdbba68b service neutron] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:50.707205 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee7226-223d-4501-b454-72c787030fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:50.707973 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee7226-223d-4501-b454-72c787030fae admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:50.711024 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee7226-223d-4501-b454-72c787030fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:50.711325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee7226-223d-4501-b454-72c787030fae admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:50.712128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cee7226-223d-4501-b454-72c787030fae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:50.720444 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cee7226-223d-4501-b454-72c787030fae admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:50.792600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 875/1724] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:02:50 2022] POST /identity/v3/users => generated 297 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:50.797723 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee7226-223d-4501-b454-72c787030fae admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:50.823623 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee7226-223d-4501-b454-72c787030fae admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:50.839336 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23ae74-c488-4bd6-aaf5-849ebd722eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:50.840416 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23ae74-c488-4bd6-aaf5-849ebd722eb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:50.840875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23ae74-c488-4bd6-aaf5-849ebd722eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:50.841165 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23ae74-c488-4bd6-aaf5-849ebd722eb1 admin admin] PATH_INFO: `/v3/users/c79ba4a88bf04becbd97c8f83748d475` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:50.844936 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df23ae74-c488-4bd6-aaf5-849ebd722eb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c79ba4a88bf04becbd97c8f83748d475)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:50.845643 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475848c-5ef7-4812-8229-dd53fdbba68b service neutron] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:50.846904 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df23ae74-c488-4bd6-aaf5-849ebd722eb1 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:50.854733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97498855-51be-4426-95e7-76cb58d21225 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:50.856168 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97498855-51be-4426-95e7-76cb58d21225 service nova] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:50.857035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97498855-51be-4426-95e7-76cb58d21225 service nova] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:50.857513 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97498855-51be-4426-95e7-76cb58d21225 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:50.872123 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97498855-51be-4426-95e7-76cb58d21225 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:50.874446 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97498855-51be-4426-95e7-76cb58d21225 service nova] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:50.914338 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 850/1725] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:50.915798 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97498855-51be-4426-95e7-76cb58d21225 service nova] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:50.950233 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97498855-51be-4426-95e7-76cb58d21225 service nova] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:50.956542 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:50.966507 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef8490-266e-49d9-9180-a23894088154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:50.967463 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-df23ae74-c488-4bd6-aaf5-849ebd722eb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c79ba4a88bf04becbd97c8f83748d475'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:50.968018 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef8490-266e-49d9-9180-a23894088154 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:50.968385 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef8490-266e-49d9-9180-a23894088154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:50.968922 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef8490-266e-49d9-9180-a23894088154 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:50.969811 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef8490-266e-49d9-9180-a23894088154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:50.974766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef8490-266e-49d9-9180-a23894088154 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:50.991663 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-df23ae74-c488-4bd6-aaf5-849ebd722eb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c79ba4a88bf04becbd97c8f83748d475'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:51.002296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 851/1726] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:51.006243 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef8490-266e-49d9-9180-a23894088154 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:51.024023 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-df23ae74-c488-4bd6-aaf5-849ebd722eb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c79ba4a88bf04becbd97c8f83748d475'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:51.045767 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-df23ae74-c488-4bd6-aaf5-849ebd722eb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c79ba4a88bf04becbd97c8f83748d475'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:51.046738 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-df23ae74-c488-4bd6-aaf5-849ebd722eb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c79ba4a88bf04becbd97c8f83748d475'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:51.054280 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef8490-266e-49d9-9180-a23894088154 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:51.081445 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 876/1727] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:02:50 2022] DELETE /identity/v3/users/c79ba4a88bf04becbd97c8f83748d475 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:51.083204 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea2756-9207-491a-8bbc-5433be9504fc None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:51.085575 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea2756-9207-491a-8bbc-5433be9504fc None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:51.085996 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea2756-9207-491a-8bbc-5433be9504fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:51.087564 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2a5c5-54d4-4da5-9353-d7bf965d74c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:51.088443 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2a5c5-54d4-4da5-9353-d7bf965d74c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:51.089276 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2a5c5-54d4-4da5-9353-d7bf965d74c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:51.090163 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2a5c5-54d4-4da5-9353-d7bf965d74c4 admin admin] PATH_INFO: `/v3/projects/3b715e90c54141d4b78fad85302d6985/users/7ad86b42cc1c4e34b055ec7ef82af3d3/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:51.136010 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2a5c5-54d4-4da5-9353-d7bf965d74c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b715e90c54141d4b78fad85302d6985, user_id=7ad86b42cc1c4e34b055ec7ef82af3d3, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:51.144083 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2a5c5-54d4-4da5-9353-d7bf965d74c4 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:51.187727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 852/1728] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:02:51 2022] PUT /identity/v3/projects/3b715e90c54141d4b78fad85302d6985/users/7ad86b42cc1c4e34b055ec7ef82af3d3/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:51.192288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2a5c5-54d4-4da5-9353-d7bf965d74c4 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:51.192818 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-86ea2756-9207-491a-8bbc-5433be9504fc None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:51.198217 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-0bf2a5c5-54d4-4da5-9353-d7bf965d74c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:51.199453 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-86ea2756-9207-491a-8bbc-5433be9504fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:51.269081 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:51.369663 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 877/1729] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:51.375651 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2a5c5-54d4-4da5-9353-d7bf965d74c4 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:51.382030 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928feb09-d5c1-4b5d-9eec-ac7727740a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:51.383059 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928feb09-d5c1-4b5d-9eec-ac7727740a08 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:51.383421 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928feb09-d5c1-4b5d-9eec-ac7727740a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:51.383759 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928feb09-d5c1-4b5d-9eec-ac7727740a08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:51.384660 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928feb09-d5c1-4b5d-9eec-ac7727740a08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:51.386918 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928feb09-d5c1-4b5d-9eec-ac7727740a08 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:51.393623 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea2756-9207-491a-8bbc-5433be9504fc None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:51.396446 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-86ea2756-9207-491a-8bbc-5433be9504fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:51.413728 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 853/1730] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:51.415719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928feb09-d5c1-4b5d-9eec-ac7727740a08 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:51.458812 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928feb09-d5c1-4b5d-9eec-ac7727740a08 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:51.476201 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8632de-110d-47da-a04f-ab1502ecb4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:51.476973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8632de-110d-47da-a04f-ab1502ecb4ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:51.477212 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8632de-110d-47da-a04f-ab1502ecb4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:51.477492 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8632de-110d-47da-a04f-ab1502ecb4ff admin admin] PATH_INFO: `/v3/projects/3b715e90c54141d4b78fad85302d6985/users/7ad86b42cc1c4e34b055ec7ef82af3d3/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:51.482436 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8632de-110d-47da-a04f-ab1502ecb4ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b715e90c54141d4b78fad85302d6985, user_id=7ad86b42cc1c4e34b055ec7ef82af3d3, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:51.486500 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8632de-110d-47da-a04f-ab1502ecb4ff admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:51.521899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 854/1731] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:02:51 2022] PUT /identity/v3/projects/3b715e90c54141d4b78fad85302d6985/users/7ad86b42cc1c4e34b055ec7ef82af3d3/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:51.523771 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c141cb0-eb29-4950-b16d-8cfb9b2d9a3d None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:51.523904 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c141cb0-eb29-4950-b16d-8cfb9b2d9a3d None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:51.524639 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c141cb0-eb29-4950-b16d-8cfb9b2d9a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:51.566851 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea2756-9207-491a-8bbc-5433be9504fc None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:51.611235 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-7c141cb0-eb29-4950-b16d-8cfb9b2d9a3d None None] MFA Rules not processed for user `7ad86b42cc1c4e34b055ec7ef82af3d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:51.618422 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-7c141cb0-eb29-4950-b16d-8cfb9b2d9a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:51.649515 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2308001-23a6-4c85-b70c-7573f446886a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:51.650210 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2308001-23a6-4c85-b70c-7573f446886a service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:51.650489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2308001-23a6-4c85-b70c-7573f446886a service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:51.650838 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2308001-23a6-4c85-b70c-7573f446886a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:51.714677 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:51.768909 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2308001-23a6-4c85-b70c-7573f446886a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:51.770629 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2308001-23a6-4c85-b70c-7573f446886a service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:51.795193 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 878/1732] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:51.811668 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 855/1733] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:51 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:02:51.822891 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2308001-23a6-4c85-b70c-7573f446886a service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:51.826812 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e2308001-23a6-4c85-b70c-7573f446886a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:51.871536 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9c35d-f35f-4a7c-a504-c65cefb68901 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:51.871826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9c35d-f35f-4a7c-a504-c65cefb68901 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:51.872105 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9c35d-f35f-4a7c-a504-c65cefb68901 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:51.874873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 856/1734] 10.222.0.39 () {54 vars in 912 bytes} [Fri Aug 5 20:02:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 20:02:51.881403 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a131b-e891-4612-b24e-640fd9cbfdc7 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:51.881681 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a131b-e891-4612-b24e-640fd9cbfdc7 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:51.881952 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a131b-e891-4612-b24e-640fd9cbfdc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:51.927462 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-339a131b-e891-4612-b24e-640fd9cbfdc7 None None] MFA Rules not processed for user `b1c75368c00e4b0cb7b3163728843479`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:51.929163 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-339a131b-e891-4612-b24e-640fd9cbfdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:51.937742 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 857/1735] 10.222.0.39 () {58 vars in 1009 bytes} [Fri Aug 5 20:02:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:51.954234 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a131b-e891-4612-b24e-640fd9cbfdc7 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:51.959632 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-339a131b-e891-4612-b24e-640fd9cbfdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:51.983057 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a131b-e891-4612-b24e-640fd9cbfdc7 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:51.990519 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2308001-23a6-4c85-b70c-7573f446886a service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:51.994389 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80a9aa-0413-45c1-8c77-32ee323575aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:51.994389 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80a9aa-0413-45c1-8c77-32ee323575aa service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:51.994389 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80a9aa-0413-45c1-8c77-32ee323575aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:51.994389 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80a9aa-0413-45c1-8c77-32ee323575aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:52.007658 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b80a9aa-0413-45c1-8c77-32ee323575aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:52.009160 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b80a9aa-0413-45c1-8c77-32ee323575aa service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:52.011381 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f2188-46d7-461a-9a97-f661b0f92d4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:52.012103 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f2188-46d7-461a-9a97-f661b0f92d4d service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:52.013565 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f2188-46d7-461a-9a97-f661b0f92d4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:52.015639 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f2188-46d7-461a-9a97-f661b0f92d4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:52.031806 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 858/1736] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:52.045372 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f2188-46d7-461a-9a97-f661b0f92d4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:52.055876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f2188-46d7-461a-9a97-f661b0f92d4d service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:52.094914 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 879/1737] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:51 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:52.235334 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a97596-9424-48e2-92d4-be7d71d48a68 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:52.235875 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a97596-9424-48e2-92d4-be7d71d48a68 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:52.236377 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a97596-9424-48e2-92d4-be7d71d48a68 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:52.238334 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 859/1738] 10.222.0.39 () {54 vars in 961 bytes} [Fri Aug 5 20:02:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 20:02:52.246118 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcc23c-5d79-44fa-9fcf-460a25bec914 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:52.246515 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcc23c-5d79-44fa-9fcf-460a25bec914 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:52.246690 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcc23c-5d79-44fa-9fcf-460a25bec914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:52.295228 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a97596-9424-48e2-92d4-be7d71d48a68 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:52.297165 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-20a97596-9424-48e2-92d4-be7d71d48a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:52.331400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-22dcc23c-5d79-44fa-9fcf-460a25bec914 None None] MFA Rules not processed for user `c61e2c954f9a49abad9f3de17ee39f63`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:52.334851 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-22dcc23c-5d79-44fa-9fcf-460a25bec914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:52.357251 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a97596-9424-48e2-92d4-be7d71d48a68 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:52.365021 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1247d28-3124-4286-8754-97f0b18114f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:52.365654 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1247d28-3124-4286-8754-97f0b18114f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:52.365935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1247d28-3124-4286-8754-97f0b18114f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:52.366212 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1247d28-3124-4286-8754-97f0b18114f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:52.371236 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-a1247d28-3124-4286-8754-97f0b18114f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:52.441101 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 880/1739] 10.222.0.39 () {58 vars in 1058 bytes} [Fri Aug 5 20:02:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:52.443548 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b213a9-ef93-4305-a0ae-a456874cbbaf None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:52.444935 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b213a9-ef93-4305-a0ae-a456874cbbaf None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:52.445581 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b213a9-ef93-4305-a0ae-a456874cbbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:52.479337 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-60b213a9-ef93-4305-a0ae-a456874cbbaf None None] MFA Rules not processed for user `293b8a62f19c45c1bf5b897c7cf40edb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:52.480877 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-60b213a9-ef93-4305-a0ae-a456874cbbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:52.510172 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1247d28-3124-4286-8754-97f0b18114f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:52.512938 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1247d28-3124-4286-8754-97f0b18114f2 service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:52.562574 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 860/1740] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:52 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:02:52.565632 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1247d28-3124-4286-8754-97f0b18114f2 service cinder] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:52.600508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:52.621406 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1247d28-3124-4286-8754-97f0b18114f2 service cinder] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:52.640470 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5aed7a-1072-4367-b85a-56be1c8f5612 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61e2c954f9a49abad9f3de17ee39f63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:52.641387 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5aed7a-1072-4367-b85a-56be1c8f5612 service placement] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:52.649700 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5aed7a-1072-4367-b85a-56be1c8f5612 service placement] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:52.650082 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5aed7a-1072-4367-b85a-56be1c8f5612 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:52.655240 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-fb5aed7a-1072-4367-b85a-56be1c8f5612 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:52.678196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 881/1741] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:52.705677 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5aed7a-1072-4367-b85a-56be1c8f5612 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:52.709094 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5aed7a-1072-4367-b85a-56be1c8f5612 service placement] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:52.769189 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 861/1742] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:52.812300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b213a9-ef93-4305-a0ae-a456874cbbaf None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:52.813891 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-60b213a9-ef93-4305-a0ae-a456874cbbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:52.855565 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b213a9-ef93-4305-a0ae-a456874cbbaf None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:52.885112 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf7fef-43e3-4ca0-a145-ec989f989412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:52.888345 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf7fef-43e3-4ca0-a145-ec989f989412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:52.890863 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf7fef-43e3-4ca0-a145-ec989f989412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:52.892459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf7fef-43e3-4ca0-a145-ec989f989412 admin admin] PATH_INFO: `/v3/users/f12b6003a1824329949e3f1ed6b43c29` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:52.896963 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf7fef-43e3-4ca0-a145-ec989f989412 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f12b6003a1824329949e3f1ed6b43c29)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:52.904072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf7fef-43e3-4ca0-a145-ec989f989412 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:52.914664 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:53.080336 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ecdf7fef-43e3-4ca0-a145-ec989f989412 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f12b6003a1824329949e3f1ed6b43c29'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:53.109214 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ecdf7fef-43e3-4ca0-a145-ec989f989412 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f12b6003a1824329949e3f1ed6b43c29'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:53.138680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ecdf7fef-43e3-4ca0-a145-ec989f989412 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f12b6003a1824329949e3f1ed6b43c29'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:53.279414 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ecdf7fef-43e3-4ca0-a145-ec989f989412 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f12b6003a1824329949e3f1ed6b43c29'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:53.285913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ecdf7fef-43e3-4ca0-a145-ec989f989412 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f12b6003a1824329949e3f1ed6b43c29'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:53.294844 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48b278-e961-45c9-a272-565eb3a82ba6 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:53.294844 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48b278-e961-45c9-a272-565eb3a82ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:53.294844 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48b278-e961-45c9-a272-565eb3a82ba6 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:53.294844 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 862/1743] 10.222.0.39 () {54 vars in 921 bytes} [Fri Aug 5 20:02:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 20:02:53.344431 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 882/1744] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:02:52 2022] DELETE /identity/v3/users/f12b6003a1824329949e3f1ed6b43c29 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:53.353961 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da255f6d-1032-4a91-a03a-9af424c36a48 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:53.354247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da255f6d-1032-4a91-a03a-9af424c36a48 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:53.354524 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da255f6d-1032-4a91-a03a-9af424c36a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:53.425164 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-da255f6d-1032-4a91-a03a-9af424c36a48 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:53.435178 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-da255f6d-1032-4a91-a03a-9af424c36a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:53.460990 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf7fef-43e3-4ca0-a145-ec989f989412 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:53.462775 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-ecdf7fef-43e3-4ca0-a145-ec989f989412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:53.608162 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf7fef-43e3-4ca0-a145-ec989f989412 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:53.615703 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d84bc7-8001-4ef6-9db3-fc612a38fbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:53.616197 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d84bc7-8001-4ef6-9db3-fc612a38fbe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:53.617253 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d84bc7-8001-4ef6-9db3-fc612a38fbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:53.617471 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d84bc7-8001-4ef6-9db3-fc612a38fbe3 admin admin] PATH_INFO: `/v3/users/7fc6b06f10574871994798ce8673ada4` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:53.619444 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d84bc7-8001-4ef6-9db3-fc612a38fbe3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fc6b06f10574871994798ce8673ada4)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:53.622059 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d84bc7-8001-4ef6-9db3-fc612a38fbe3 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:53.660616 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 863/1745] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:53.667951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d533c5-dfe8-4b01-8680-5b4a8160db4b None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:53.671733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d533c5-dfe8-4b01-8680-5b4a8160db4b None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:53.672551 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d533c5-dfe8-4b01-8680-5b4a8160db4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:53.731679 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-33d84bc7-8001-4ef6-9db3-fc612a38fbe3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fc6b06f10574871994798ce8673ada4'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:53.735525 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-90d533c5-dfe8-4b01-8680-5b4a8160db4b None None] MFA Rules not processed for user `c61e2c954f9a49abad9f3de17ee39f63`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:53.740676 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-90d533c5-dfe8-4b01-8680-5b4a8160db4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:53.761804 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-33d84bc7-8001-4ef6-9db3-fc612a38fbe3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fc6b06f10574871994798ce8673ada4'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:53.773035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-33d84bc7-8001-4ef6-9db3-fc612a38fbe3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fc6b06f10574871994798ce8673ada4'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:53.846931 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-33d84bc7-8001-4ef6-9db3-fc612a38fbe3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fc6b06f10574871994798ce8673ada4'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:53.849107 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-33d84bc7-8001-4ef6-9db3-fc612a38fbe3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fc6b06f10574871994798ce8673ada4'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:53.880743 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:53.889966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 883/1746] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:02:53 2022] DELETE /identity/v3/users/7fc6b06f10574871994798ce8673ada4 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:53.889966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d84bc7-8001-4ef6-9db3-fc612a38fbe3 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:53.890508 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-33d84bc7-8001-4ef6-9db3-fc612a38fbe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:53.936168 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 864/1747] 10.222.0.39 () {58 vars in 1018 bytes} [Fri Aug 5 20:02:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:53.938253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99cbea-e6fc-4127-9ea3-4c1c2e552455 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:53.938555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99cbea-e6fc-4127-9ea3-4c1c2e552455 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:53.938826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99cbea-e6fc-4127-9ea3-4c1c2e552455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:54.017733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-ee99cbea-e6fc-4127-9ea3-4c1c2e552455 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:54.018987 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-ee99cbea-e6fc-4127-9ea3-4c1c2e552455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:54.104026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d84bc7-8001-4ef6-9db3-fc612a38fbe3 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:54.123654 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cefbac-4781-463f-8a60-cd28bc323725 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:54.124615 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cefbac-4781-463f-8a60-cd28bc323725 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:54.125117 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cefbac-4781-463f-8a60-cd28bc323725 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:54.127879 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cefbac-4781-463f-8a60-cd28bc323725 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:54.136428 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-40cefbac-4781-463f-8a60-cd28bc323725 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:54.355612 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 865/1748] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:54.359058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99cbea-e6fc-4127-9ea3-4c1c2e552455 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:54.368158 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-ee99cbea-e6fc-4127-9ea3-4c1c2e552455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:54.374341 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cefbac-4781-463f-8a60-cd28bc323725 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:54.375777 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cefbac-4781-463f-8a60-cd28bc323725 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:54.447622 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 884/1749] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:54.500242 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:54.604038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cefbac-4781-463f-8a60-cd28bc323725 service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:54.605614 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-40cefbac-4781-463f-8a60-cd28bc323725 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:54.626697 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cefbac-4781-463f-8a60-cd28bc323725 service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:54.642847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99cbea-e6fc-4127-9ea3-4c1c2e552455 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:54.663621 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a799be-5193-4bf3-a78b-f6eb5021a082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:54.663858 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a799be-5193-4bf3-a78b-f6eb5021a082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:54.664105 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a799be-5193-4bf3-a78b-f6eb5021a082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:54.664390 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a799be-5193-4bf3-a78b-f6eb5021a082 admin admin] PATH_INFO: `/v3/users/4da8ef6d2bc141c79a068a8f9a6a3254` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:54.672211 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a799be-5193-4bf3-a78b-f6eb5021a082 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4da8ef6d2bc141c79a068a8f9a6a3254)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:54.675914 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d3f00-1505-491c-8268-ec7ce3d0627a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61e2c954f9a49abad9f3de17ee39f63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:54.678590 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a799be-5193-4bf3-a78b-f6eb5021a082 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:54.678993 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d3f00-1505-491c-8268-ec7ce3d0627a service placement] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:54.681305 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d3f00-1505-491c-8268-ec7ce3d0627a service placement] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:54.682099 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d3f00-1505-491c-8268-ec7ce3d0627a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:54.706369 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d3f00-1505-491c-8268-ec7ce3d0627a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:54.711635 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d3f00-1505-491c-8268-ec7ce3d0627a service placement] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:54.743614 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 866/1750] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:54.747106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d3f00-1505-491c-8268-ec7ce3d0627a service placement] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:54.772268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d3f00-1505-491c-8268-ec7ce3d0627a service placement] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:54.781648 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13214d41-4591-42d7-b7f0-82dda24d2d53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:54.781648 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13214d41-4591-42d7-b7f0-82dda24d2d53 service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:54.781648 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13214d41-4591-42d7-b7f0-82dda24d2d53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:54.781648 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13214d41-4591-42d7-b7f0-82dda24d2d53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:54.821444 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13214d41-4591-42d7-b7f0-82dda24d2d53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:54.824840 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13214d41-4591-42d7-b7f0-82dda24d2d53 service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:54.842448 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 867/1751] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:54.890644 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-f7a799be-5193-4bf3-a78b-f6eb5021a082 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4da8ef6d2bc141c79a068a8f9a6a3254'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:54.914718 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-f7a799be-5193-4bf3-a78b-f6eb5021a082 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4da8ef6d2bc141c79a068a8f9a6a3254'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:54.953358 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-f7a799be-5193-4bf3-a78b-f6eb5021a082 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4da8ef6d2bc141c79a068a8f9a6a3254'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:54.993052 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-f7a799be-5193-4bf3-a78b-f6eb5021a082 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4da8ef6d2bc141c79a068a8f9a6a3254'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:54.995109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-f7a799be-5193-4bf3-a78b-f6eb5021a082 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4da8ef6d2bc141c79a068a8f9a6a3254'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:55.009226 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 885/1752] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:02:54 2022] DELETE /identity/v3/users/4da8ef6d2bc141c79a068a8f9a6a3254 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:02:55.013731 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253210a4-f115-4c68-834b-a068c35d054e None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:55.014014 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253210a4-f115-4c68-834b-a068c35d054e None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:55.014289 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253210a4-f115-4c68-834b-a068c35d054e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:55.073420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-253210a4-f115-4c68-834b-a068c35d054e None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:55.074661 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-253210a4-f115-4c68-834b-a068c35d054e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:55.267122 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 868/1753] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:55.318428 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a799be-5193-4bf3-a78b-f6eb5021a082 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:55.321518 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-f7a799be-5193-4bf3-a78b-f6eb5021a082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:55.377878 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:55.507399 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a799be-5193-4bf3-a78b-f6eb5021a082 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:55.523369 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49bd1e6-df7a-48fc-89ee-4ebcc20b52dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:55.524023 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bd1e6-df7a-48fc-89ee-4ebcc20b52dc service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:55.524306 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bd1e6-df7a-48fc-89ee-4ebcc20b52dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:55.524601 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bd1e6-df7a-48fc-89ee-4ebcc20b52dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:55.536821 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49bd1e6-df7a-48fc-89ee-4ebcc20b52dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:55.538627 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49bd1e6-df7a-48fc-89ee-4ebcc20b52dc service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:55.559545 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 886/1754] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:55.701784 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253210a4-f115-4c68-834b-a068c35d054e None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:55.704267 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-253210a4-f115-4c68-834b-a068c35d054e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:55.740862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461daac-3a4f-4662-a458-7f75eec7724c None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:55.741537 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461daac-3a4f-4662-a458-7f75eec7724c None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:55.742080 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461daac-3a4f-4662-a458-7f75eec7724c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:55.781794 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253210a4-f115-4c68-834b-a068c35d054e None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:55.797298 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b5ebe-6311-445b-8bf8-f4391213a45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:55.798868 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b5ebe-6311-445b-8bf8-f4391213a45d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:55.799254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b5ebe-6311-445b-8bf8-f4391213a45d admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:55.799426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-b461daac-3a4f-4662-a458-7f75eec7724c None None] MFA Rules not processed for user `1afe13c8fbb1468b8ef3d0d0c671208d`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:55.799714 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b5ebe-6311-445b-8bf8-f4391213a45d admin admin] PATH_INFO: `/v3/projects/792c790017474d1399dfe323d2a8a299` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:55.801886 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6b5ebe-6311-445b-8bf8-f4391213a45d admin admin] RBAC: Authorizing `identity:delete_project(project_id=792c790017474d1399dfe323d2a8a299)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:55.803574 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-b461daac-3a4f-4662-a458-7f75eec7724c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:55.806490 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6b5ebe-6311-445b-8bf8-f4391213a45d admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:55.917305 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-2e6b5ebe-6311-445b-8bf8-f4391213a45d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '792c790017474d1399dfe323d2a8a299', 'request_id': 'req-2e6b5ebe-6311-445b-8bf8-f4391213a45d'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:55.938760 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-2e6b5ebe-6311-445b-8bf8-f4391213a45d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '792c790017474d1399dfe323d2a8a299', 'request_id': 'req-2e6b5ebe-6311-445b-8bf8-f4391213a45d'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:55.946071 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-2e6b5ebe-6311-445b-8bf8-f4391213a45d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '792c790017474d1399dfe323d2a8a299', 'request_id': 'req-2e6b5ebe-6311-445b-8bf8-f4391213a45d'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:55.948765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 887/1755] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:55.974585 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-2e6b5ebe-6311-445b-8bf8-f4391213a45d admin admin] The token cache is being invalidate because project 792c790017474d1399dfe323d2a8a299 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:02:55.974986 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-2e6b5ebe-6311-445b-8bf8-f4391213a45d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:55.976220 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-2e6b5ebe-6311-445b-8bf8-f4391213a45d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:55.981004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 869/1756] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:02:55 2022] DELETE /identity/v3/projects/792c790017474d1399dfe323d2a8a299 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:55.983246 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b461daac-3a4f-4662-a458-7f75eec7724c None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:56.003429 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b461daac-3a4f-4662-a458-7f75eec7724c None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:56.152127 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a20b9f-b68c-4df2-9c74-12a157278f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:56.153261 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a20b9f-b68c-4df2-9c74-12a157278f89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:56.153676 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a20b9f-b68c-4df2-9c74-12a157278f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:56.154129 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a20b9f-b68c-4df2-9c74-12a157278f89 admin admin] PATH_INFO: `/v3/users/9762ad5809074941a3257f148d01e560` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:56.157101 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a20b9f-b68c-4df2-9c74-12a157278f89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9762ad5809074941a3257f148d01e560)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:56.159555 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a20b9f-b68c-4df2-9c74-12a157278f89 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:56.276953 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-26a20b9f-b68c-4df2-9c74-12a157278f89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9762ad5809074941a3257f148d01e560'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:56.295766 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-26a20b9f-b68c-4df2-9c74-12a157278f89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9762ad5809074941a3257f148d01e560'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:56.308854 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-26a20b9f-b68c-4df2-9c74-12a157278f89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9762ad5809074941a3257f148d01e560'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:56.317182 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:56.330519 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-26a20b9f-b68c-4df2-9c74-12a157278f89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9762ad5809074941a3257f148d01e560'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:56.331441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-26a20b9f-b68c-4df2-9c74-12a157278f89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9762ad5809074941a3257f148d01e560'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:56.343731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 888/1757] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:02:55 2022] DELETE /identity/v3/users/9762ad5809074941a3257f148d01e560 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:56.455476 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b5ebe-6311-445b-8bf8-f4391213a45d admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:56.458319 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-2e6b5ebe-6311-445b-8bf8-f4391213a45d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:56.498685 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b5ebe-6311-445b-8bf8-f4391213a45d admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:56.597308 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:56.792803 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b6349-a235-439e-a72d-4e4876449b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:56.795432 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b6349-a235-439e-a72d-4e4876449b4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:56.795741 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b6349-a235-439e-a72d-4e4876449b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:56.798269 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b6349-a235-439e-a72d-4e4876449b4f admin admin] PATH_INFO: `/v3/projects/fbd1312a3fb2498488a4525c15e26f2d` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:56.802887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92b6349-a235-439e-a72d-4e4876449b4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbd1312a3fb2498488a4525c15e26f2d)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:56.815428 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92b6349-a235-439e-a72d-4e4876449b4f admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:57.008764 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a20b9f-b68c-4df2-9c74-12a157278f89 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:57.010667 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-26a20b9f-b68c-4df2-9c74-12a157278f89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:57.036836 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-d92b6349-a235-439e-a72d-4e4876449b4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbd1312a3fb2498488a4525c15e26f2d', 'request_id': 'req-d92b6349-a235-439e-a72d-4e4876449b4f'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:57.050082 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a20b9f-b68c-4df2-9c74-12a157278f89 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:57.069535 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-d92b6349-a235-439e-a72d-4e4876449b4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbd1312a3fb2498488a4525c15e26f2d', 'request_id': 'req-d92b6349-a235-439e-a72d-4e4876449b4f'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:57.077813 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-d92b6349-a235-439e-a72d-4e4876449b4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbd1312a3fb2498488a4525c15e26f2d', 'request_id': 'req-d92b6349-a235-439e-a72d-4e4876449b4f'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:57.112365 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-d92b6349-a235-439e-a72d-4e4876449b4f admin admin] The token cache is being invalidate because project fbd1312a3fb2498488a4525c15e26f2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:02:57.113158 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-d92b6349-a235-439e-a72d-4e4876449b4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:57.116919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-d92b6349-a235-439e-a72d-4e4876449b4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:57.124646 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 870/1758] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:02:56 2022] DELETE /identity/v3/projects/fbd1312a3fb2498488a4525c15e26f2d => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:57.137465 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b6349-a235-439e-a72d-4e4876449b4f admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:57.143325 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-d92b6349-a235-439e-a72d-4e4876449b4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:57.292028 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f7e804-4eef-4b72-8092-420ac44e0634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:57.292722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7e804-4eef-4b72-8092-420ac44e0634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:57.293704 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7e804-4eef-4b72-8092-420ac44e0634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:57.294126 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7e804-4eef-4b72-8092-420ac44e0634 admin admin] PATH_INFO: `/v3/projects/de7f4f5adc804c17834f22ab4fb0c29d` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:57.297215 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f7e804-4eef-4b72-8092-420ac44e0634 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de7f4f5adc804c17834f22ab4fb0c29d)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:57.298135 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f7e804-4eef-4b72-8092-420ac44e0634 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:57.403266 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b6349-a235-439e-a72d-4e4876449b4f admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:57.419441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-a6f7e804-4eef-4b72-8092-420ac44e0634 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de7f4f5adc804c17834f22ab4fb0c29d', 'request_id': 'req-a6f7e804-4eef-4b72-8092-420ac44e0634'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:57.437272 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-a6f7e804-4eef-4b72-8092-420ac44e0634 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de7f4f5adc804c17834f22ab4fb0c29d', 'request_id': 'req-a6f7e804-4eef-4b72-8092-420ac44e0634'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:57.456402 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-a6f7e804-4eef-4b72-8092-420ac44e0634 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de7f4f5adc804c17834f22ab4fb0c29d', 'request_id': 'req-a6f7e804-4eef-4b72-8092-420ac44e0634'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:57.484996 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-a6f7e804-4eef-4b72-8092-420ac44e0634 admin admin] The token cache is being invalidate because project de7f4f5adc804c17834f22ab4fb0c29d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:02:57.485310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-a6f7e804-4eef-4b72-8092-420ac44e0634 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:57.487733 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-a6f7e804-4eef-4b72-8092-420ac44e0634 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:57.495604 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 889/1759] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:02:57 2022] DELETE /identity/v3/projects/de7f4f5adc804c17834f22ab4fb0c29d => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:57.501346 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbf304-0652-4277-9dfa-6394c1190deb None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:57.501634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbf304-0652-4277-9dfa-6394c1190deb None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:57.501913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbf304-0652-4277-9dfa-6394c1190deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:57.553836 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a403ba-4a0c-4232-9e32-3a6c78587405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:57.558321 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a403ba-4a0c-4232-9e32-3a6c78587405 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:57.558613 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a403ba-4a0c-4232-9e32-3a6c78587405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:57.558894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a403ba-4a0c-4232-9e32-3a6c78587405 admin admin] PATH_INFO: `/v3/users/a62039722920453b8cc6f8befc0aaa45` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:57.559687 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-d3cbf304-0652-4277-9dfa-6394c1190deb None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:57.561197 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-d3cbf304-0652-4277-9dfa-6394c1190deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:57.563340 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a403ba-4a0c-4232-9e32-3a6c78587405 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a62039722920453b8cc6f8befc0aaa45)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:57.570068 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a403ba-4a0c-4232-9e32-3a6c78587405 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:57.573266 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 890/1760] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:57.574782 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567de82-edca-4adb-b63a-5ac7e80ab47a None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:57.575069 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567de82-edca-4adb-b63a-5ac7e80ab47a None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:57.575338 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567de82-edca-4adb-b63a-5ac7e80ab47a None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:57.576966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 891/1761] 10.222.0.39 () {54 vars in 921 bytes} [Fri Aug 5 20:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 20:02:57.582599 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96460469-296a-42bd-b816-25690155db34 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:57.583695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96460469-296a-42bd-b816-25690155db34 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:57.583695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96460469-296a-42bd-b816-25690155db34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:57.590826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:57.609512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-96460469-296a-42bd-b816-25690155db34 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:57.611225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-96460469-296a-42bd-b816-25690155db34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:57.636223 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 892/1762] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:02:57.640664 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96460469-296a-42bd-b816-25690155db34 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:57.666182 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96460469-296a-42bd-b816-25690155db34 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:57.671990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-00a403ba-4a0c-4232-9e32-3a6c78587405 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a62039722920453b8cc6f8befc0aaa45'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:57.683228 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7663e9b-9663-4fca-b2e1-af211a1aca42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:57.684358 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7663e9b-9663-4fca-b2e1-af211a1aca42 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:57.684358 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7663e9b-9663-4fca-b2e1-af211a1aca42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:57.684540 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7663e9b-9663-4fca-b2e1-af211a1aca42 admin admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:57.685209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7663e9b-9663-4fca-b2e1-af211a1aca42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:02:57.685520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7663e9b-9663-4fca-b2e1-af211a1aca42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:57.687430 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7663e9b-9663-4fca-b2e1-af211a1aca42 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:57.695096 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-00a403ba-4a0c-4232-9e32-3a6c78587405 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a62039722920453b8cc6f8befc0aaa45'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:57.715019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-00a403ba-4a0c-4232-9e32-3a6c78587405 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a62039722920453b8cc6f8befc0aaa45'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:57.715789 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-00a403ba-4a0c-4232-9e32-3a6c78587405 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a62039722920453b8cc6f8befc0aaa45'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:57.719566 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 893/1763] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 20:02:57.719566 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c30e1a-e59b-4a49-85ac-cf95d729728c None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:57.719566 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c30e1a-e59b-4a49-85ac-cf95d729728c None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:57.719566 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c30e1a-e59b-4a49-85ac-cf95d729728c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:57.728018 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-00a403ba-4a0c-4232-9e32-3a6c78587405 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a62039722920453b8cc6f8befc0aaa45'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:57.739920 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 871/1764] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:02:57 2022] DELETE /identity/v3/users/a62039722920453b8cc6f8befc0aaa45 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:57.741263 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09114843-bd44-4750-a06a-d054ed5dc99a None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:57.741552 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09114843-bd44-4750-a06a-d054ed5dc99a None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:57.741830 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09114843-bd44-4750-a06a-d054ed5dc99a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:57.769961 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-09114843-bd44-4750-a06a-d054ed5dc99a None None] MFA Rules not processed for user `c61e2c954f9a49abad9f3de17ee39f63`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:57.772047 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-09114843-bd44-4750-a06a-d054ed5dc99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:57.845624 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-87c30e1a-e59b-4a49-85ac-cf95d729728c None None] MFA Rules not processed for user `3488a7d2258242c0859a105c1d162767`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:57.846937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-87c30e1a-e59b-4a49-85ac-cf95d729728c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:57.881913 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 872/1765] 10.222.0.39 () {58 vars in 1018 bytes} [Fri Aug 5 20:02:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:57.883350 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09114843-bd44-4750-a06a-d054ed5dc99a None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:57.884942 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-09114843-bd44-4750-a06a-d054ed5dc99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:57.888856 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:58.000978 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09114843-bd44-4750-a06a-d054ed5dc99a None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:58.001727 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 894/1766] 10.222.0.39 () {58 vars in 1009 bytes} [Fri Aug 5 20:02:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:58.002582 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c30e1a-e59b-4a49-85ac-cf95d729728c None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:58.003963 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-87c30e1a-e59b-4a49-85ac-cf95d729728c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:58.022233 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c30e1a-e59b-4a49-85ac-cf95d729728c None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:58.023264 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f9c83-3dce-4eeb-bc57-58e92b699ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:58.024008 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f9c83-3dce-4eeb-bc57-58e92b699ec5 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:58.024601 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f9c83-3dce-4eeb-bc57-58e92b699ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:58.025212 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f9c83-3dce-4eeb-bc57-58e92b699ec5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:58.026092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f9c83-3dce-4eeb-bc57-58e92b699ec5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:58.028387 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f9c83-3dce-4eeb-bc57-58e92b699ec5 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:58.037367 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5150a1a1-2353-46e8-af22-e41b6e15a4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:58.038278 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150a1a1-2353-46e8-af22-e41b6e15a4b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:58.038558 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150a1a1-2353-46e8-af22-e41b6e15a4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:58.038821 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150a1a1-2353-46e8-af22-e41b6e15a4b7 admin admin] PATH_INFO: `/v3/projects/f1d08717b2e340dd8a1bb6d56310ae53` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:58.050003 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5150a1a1-2353-46e8-af22-e41b6e15a4b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1d08717b2e340dd8a1bb6d56310ae53)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:58.052906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5150a1a1-2353-46e8-af22-e41b6e15a4b7 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:58.149234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5150a1a1-2353-46e8-af22-e41b6e15a4b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1d08717b2e340dd8a1bb6d56310ae53', 'request_id': 'req-5150a1a1-2353-46e8-af22-e41b6e15a4b7'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:58.157445 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5150a1a1-2353-46e8-af22-e41b6e15a4b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1d08717b2e340dd8a1bb6d56310ae53', 'request_id': 'req-5150a1a1-2353-46e8-af22-e41b6e15a4b7'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:58.174598 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5150a1a1-2353-46e8-af22-e41b6e15a4b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1d08717b2e340dd8a1bb6d56310ae53', 'request_id': 'req-5150a1a1-2353-46e8-af22-e41b6e15a4b7'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:58.175897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 873/1767] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:02:57 2022] POST /identity/v3/projects => generated 405 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:58.177225 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f9c83-3dce-4eeb-bc57-58e92b699ec5 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:58.179615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-3a5f9c83-3dce-4eeb-bc57-58e92b699ec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:58.190136 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:58.198663 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5150a1a1-2353-46e8-af22-e41b6e15a4b7 admin admin] The token cache is being invalidate because project f1d08717b2e340dd8a1bb6d56310ae53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:02:58.199629 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5150a1a1-2353-46e8-af22-e41b6e15a4b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:58.202988 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5150a1a1-2353-46e8-af22-e41b6e15a4b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:58.206486 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 895/1768] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:02:58 2022] DELETE /identity/v3/projects/f1d08717b2e340dd8a1bb6d56310ae53 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:58.214773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5150a1a1-2353-46e8-af22-e41b6e15a4b7 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:58.235597 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5150a1a1-2353-46e8-af22-e41b6e15a4b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:58.360811 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f9c83-3dce-4eeb-bc57-58e92b699ec5 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:58.369655 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc0379-dbbb-46f4-8403-69d09e1f88f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:58.371825 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc0379-dbbb-46f4-8403-69d09e1f88f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:58.372329 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc0379-dbbb-46f4-8403-69d09e1f88f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:58.372663 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc0379-dbbb-46f4-8403-69d09e1f88f6 admin admin] PATH_INFO: `/v3/users/13a8b01944e241bfb9fd9c2e30c2fc43` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:58.378088 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc0379-dbbb-46f4-8403-69d09e1f88f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13a8b01944e241bfb9fd9c2e30c2fc43)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:58.379518 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc0379-dbbb-46f4-8403-69d09e1f88f6 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:58.426293 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5150a1a1-2353-46e8-af22-e41b6e15a4b7 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:58.486301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-87dc0379-dbbb-46f4-8403-69d09e1f88f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13a8b01944e241bfb9fd9c2e30c2fc43'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:58.496492 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:58.503036 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-87dc0379-dbbb-46f4-8403-69d09e1f88f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13a8b01944e241bfb9fd9c2e30c2fc43'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:58.527491 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b870d7-3b29-4d5d-92df-bb994c53c519 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61e2c954f9a49abad9f3de17ee39f63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:58.528133 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b870d7-3b29-4d5d-92df-bb994c53c519 service placement] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:58.528421 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b870d7-3b29-4d5d-92df-bb994c53c519 service placement] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:58.529662 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b870d7-3b29-4d5d-92df-bb994c53c519 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:58.543280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-23b870d7-3b29-4d5d-92df-bb994c53c519 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:58.544597 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-87dc0379-dbbb-46f4-8403-69d09e1f88f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13a8b01944e241bfb9fd9c2e30c2fc43'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:58.546076 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-87dc0379-dbbb-46f4-8403-69d09e1f88f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13a8b01944e241bfb9fd9c2e30c2fc43'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:58.561015 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-87dc0379-dbbb-46f4-8403-69d09e1f88f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13a8b01944e241bfb9fd9c2e30c2fc43'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:58.573952 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 874/1769] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:02:58 2022] DELETE /identity/v3/users/13a8b01944e241bfb9fd9c2e30c2fc43 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:58.573952 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc0379-dbbb-46f4-8403-69d09e1f88f6 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:58.576447 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-87dc0379-dbbb-46f4-8403-69d09e1f88f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:58.582192 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b870d7-3b29-4d5d-92df-bb994c53c519 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:58.587144 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b870d7-3b29-4d5d-92df-bb994c53c519 service placement] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:58.628647 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 896/1770] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:58.632661 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338e382-9464-4008-957a-ba33b516b42e None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:58.632954 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338e382-9464-4008-957a-ba33b516b42e None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:58.633254 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338e382-9464-4008-957a-ba33b516b42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:58.689307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-b338e382-9464-4008-957a-ba33b516b42e None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:58.691941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-b338e382-9464-4008-957a-ba33b516b42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:58.789337 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc0379-dbbb-46f4-8403-69d09e1f88f6 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:58.811146 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca1c02-e29d-4e34-b82f-afe032276b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:58.811766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca1c02-e29d-4e34-b82f-afe032276b33 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:58.812049 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca1c02-e29d-4e34-b82f-afe032276b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:58.812329 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca1c02-e29d-4e34-b82f-afe032276b33 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:58.813116 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca1c02-e29d-4e34-b82f-afe032276b33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:58.815505 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca1c02-e29d-4e34-b82f-afe032276b33 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:58.860102 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 875/1771] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:02:58 2022] POST /identity/v3/users => generated 303 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:02:58.863668 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca1c02-e29d-4e34-b82f-afe032276b33 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:58.868738 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-b4ca1c02-e29d-4e34-b82f-afe032276b33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:58.890483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 897/1772] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:58.891497 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b338e382-9464-4008-957a-ba33b516b42e None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:58.897702 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-b338e382-9464-4008-957a-ba33b516b42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:58.924806 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b338e382-9464-4008-957a-ba33b516b42e None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:58.931952 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:58.940142 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311e54d-6b93-4210-ad41-254e3b7f6828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:58.940928 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311e54d-6b93-4210-ad41-254e3b7f6828 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:58.942964 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311e54d-6b93-4210-ad41-254e3b7f6828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:58.943509 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311e54d-6b93-4210-ad41-254e3b7f6828 admin admin] PATH_INFO: `/v3/projects/c86fa13f44f14576983128e4bda934e7` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:58.946901 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3311e54d-6b93-4210-ad41-254e3b7f6828 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c86fa13f44f14576983128e4bda934e7)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:58.950126 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3311e54d-6b93-4210-ad41-254e3b7f6828 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:59.067110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-3311e54d-6b93-4210-ad41-254e3b7f6828 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c86fa13f44f14576983128e4bda934e7', 'request_id': 'req-3311e54d-6b93-4210-ad41-254e3b7f6828'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:59.084173 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-3311e54d-6b93-4210-ad41-254e3b7f6828 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c86fa13f44f14576983128e4bda934e7', 'request_id': 'req-3311e54d-6b93-4210-ad41-254e3b7f6828'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:59.101973 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-3311e54d-6b93-4210-ad41-254e3b7f6828 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c86fa13f44f14576983128e4bda934e7', 'request_id': 'req-3311e54d-6b93-4210-ad41-254e3b7f6828'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:59.127367 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-3311e54d-6b93-4210-ad41-254e3b7f6828 admin admin] The token cache is being invalidate because project c86fa13f44f14576983128e4bda934e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:02:59.127703 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-3311e54d-6b93-4210-ad41-254e3b7f6828 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:59.130237 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-3311e54d-6b93-4210-ad41-254e3b7f6828 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:59.135609 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 898/1773] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:02:58 2022] DELETE /identity/v3/projects/c86fa13f44f14576983128e4bda934e7 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:59.137013 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311e54d-6b93-4210-ad41-254e3b7f6828 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:59.140429 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-3311e54d-6b93-4210-ad41-254e3b7f6828 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:59.167525 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca1c02-e29d-4e34-b82f-afe032276b33 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:59.306981 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311e54d-6b93-4210-ad41-254e3b7f6828 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:59.313986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0306b7ae-7524-4db9-af7f-bc6f1271d808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:59.314627 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306b7ae-7524-4db9-af7f-bc6f1271d808 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:59.314915 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306b7ae-7524-4db9-af7f-bc6f1271d808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:59.315193 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306b7ae-7524-4db9-af7f-bc6f1271d808 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:59.316158 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0306b7ae-7524-4db9-af7f-bc6f1271d808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:59.318297 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0306b7ae-7524-4db9-af7f-bc6f1271d808 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:59.321029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:59.329468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce3021d-f9de-4be3-8d7d-b825aa33b376 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91d3d43cb0bd4ca3b8e8f10e29cc421c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:59.330118 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3021d-f9de-4be3-8d7d-b825aa33b376 service swift] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:59.330455 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3021d-f9de-4be3-8d7d-b825aa33b376 service swift] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:59.330668 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3021d-f9de-4be3-8d7d-b825aa33b376 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:59.335975 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 899/1774] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:59.338293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-cce3021d-f9de-4be3-8d7d-b825aa33b376 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:59.338434 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91991a9-45ce-48ec-aa62-34e92d9125a6 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:59.338434 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91991a9-45ce-48ec-aa62-34e92d9125a6 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:59.338719 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91991a9-45ce-48ec-aa62-34e92d9125a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:59.386507 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-e91991a9-45ce-48ec-aa62-34e92d9125a6 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:59.390076 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e91991a9-45ce-48ec-aa62-34e92d9125a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:59.427974 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 900/1775] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:02:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:59.427974 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91991a9-45ce-48ec-aa62-34e92d9125a6 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:59.429362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e91991a9-45ce-48ec-aa62-34e92d9125a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:59.467191 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91991a9-45ce-48ec-aa62-34e92d9125a6 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:59.491562 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eadda3-f59a-4244-aa3a-07c4065cc977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:59.491854 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eadda3-f59a-4244-aa3a-07c4065cc977 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:59.492152 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eadda3-f59a-4244-aa3a-07c4065cc977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:59.492398 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eadda3-f59a-4244-aa3a-07c4065cc977 admin admin] PATH_INFO: `/v3/users/7ad86b42cc1c4e34b055ec7ef82af3d3` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:59.494234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eadda3-f59a-4244-aa3a-07c4065cc977 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ad86b42cc1c4e34b055ec7ef82af3d3)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:59.496170 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eadda3-f59a-4244-aa3a-07c4065cc977 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:59.530480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce3021d-f9de-4be3-8d7d-b825aa33b376 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:59.543294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce3021d-f9de-4be3-8d7d-b825aa33b376 service swift] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:59.562466 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 876/1776] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Aug 5 20:02:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 702 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 20:02:59.564404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce3021d-f9de-4be3-8d7d-b825aa33b376 service swift] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:59.598442 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce3021d-f9de-4be3-8d7d-b825aa33b376 service swift] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:59.625210 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026e857-6a79-4b6e-95e6-4f739679007f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:59.625842 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026e857-6a79-4b6e-95e6-4f739679007f admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:59.626117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026e857-6a79-4b6e-95e6-4f739679007f admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:59.626386 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026e857-6a79-4b6e-95e6-4f739679007f admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:59.638392 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0026e857-6a79-4b6e-95e6-4f739679007f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:59.652671 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0026e857-6a79-4b6e-95e6-4f739679007f admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:59.685145 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-66eadda3-f59a-4244-aa3a-07c4065cc977 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ad86b42cc1c4e34b055ec7ef82af3d3'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:59.712207 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 877/1777] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:02:59.714303 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026e857-6a79-4b6e-95e6-4f739679007f admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:59.727469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-66eadda3-f59a-4244-aa3a-07c4065cc977 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ad86b42cc1c4e34b055ec7ef82af3d3'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:59.759038 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026e857-6a79-4b6e-95e6-4f739679007f admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:59.772797 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-66eadda3-f59a-4244-aa3a-07c4065cc977 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ad86b42cc1c4e34b055ec7ef82af3d3'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:59.776372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5822d201-2b54-4960-877b-973c03779547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:59.777163 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822d201-2b54-4960-877b-973c03779547 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:59.777447 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822d201-2b54-4960-877b-973c03779547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:59.777912 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822d201-2b54-4960-877b-973c03779547 admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:59.778703 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5822d201-2b54-4960-877b-973c03779547 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:02:59.780476 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5822d201-2b54-4960-877b-973c03779547 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:59.782474 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5822d201-2b54-4960-877b-973c03779547 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:59.833415 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 878/1778] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:02:59.834605 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5822d201-2b54-4960-877b-973c03779547 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:59.849847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-5822d201-2b54-4960-877b-973c03779547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:59.898792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-66eadda3-f59a-4244-aa3a-07c4065cc977 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ad86b42cc1c4e34b055ec7ef82af3d3'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:59.903554 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-66eadda3-f59a-4244-aa3a-07c4065cc977 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ad86b42cc1c4e34b055ec7ef82af3d3'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:59.914527 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5822d201-2b54-4960-877b-973c03779547 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:59.930817 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 901/1779] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:02:59 2022] DELETE /identity/v3/users/7ad86b42cc1c4e34b055ec7ef82af3d3 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:59.936832 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eadda3-f59a-4244-aa3a-07c4065cc977 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:59.945770 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-66eadda3-f59a-4244-aa3a-07c4065cc977 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:59.960219 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77d99b-841c-4e5e-8976-89bfe3b6a390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:59.963114 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77d99b-841c-4e5e-8976-89bfe3b6a390 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:59.965884 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77d99b-841c-4e5e-8976-89bfe3b6a390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:59.967896 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77d99b-841c-4e5e-8976-89bfe3b6a390 admin admin] PATH_INFO: `/v3/projects/a945deeae8dd4371aab957d526ffe1f7` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:59.973732 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a77d99b-841c-4e5e-8976-89bfe3b6a390 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a945deeae8dd4371aab957d526ffe1f7)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:59.979356 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a77d99b-841c-4e5e-8976-89bfe3b6a390 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:00.031023 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:00.166030 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-0a77d99b-841c-4e5e-8976-89bfe3b6a390 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a945deeae8dd4371aab957d526ffe1f7', 'request_id': 'req-0a77d99b-841c-4e5e-8976-89bfe3b6a390'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:00.197917 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-0a77d99b-841c-4e5e-8976-89bfe3b6a390 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a945deeae8dd4371aab957d526ffe1f7', 'request_id': 'req-0a77d99b-841c-4e5e-8976-89bfe3b6a390'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:00.232366 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-0a77d99b-841c-4e5e-8976-89bfe3b6a390 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a945deeae8dd4371aab957d526ffe1f7', 'request_id': 'req-0a77d99b-841c-4e5e-8976-89bfe3b6a390'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:00.239867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eadda3-f59a-4244-aa3a-07c4065cc977 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:00.298523 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-0a77d99b-841c-4e5e-8976-89bfe3b6a390 admin admin] The token cache is being invalidate because project a945deeae8dd4371aab957d526ffe1f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:03:00.298700 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-0a77d99b-841c-4e5e-8976-89bfe3b6a390 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:00.299605 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-0a77d99b-841c-4e5e-8976-89bfe3b6a390 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:00.303600 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 879/1780] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:02:59 2022] DELETE /identity/v3/projects/a945deeae8dd4371aab957d526ffe1f7 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:00.304796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77d99b-841c-4e5e-8976-89bfe3b6a390 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:00.306955 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-0a77d99b-841c-4e5e-8976-89bfe3b6a390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:00.479056 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77d99b-841c-4e5e-8976-89bfe3b6a390 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:00.494548 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712d6af-0f25-46f1-8937-8e4fbaac1d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:00.495113 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712d6af-0f25-46f1-8937-8e4fbaac1d21 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:00.495622 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712d6af-0f25-46f1-8937-8e4fbaac1d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:00.495952 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712d6af-0f25-46f1-8937-8e4fbaac1d21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:00.496843 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7712d6af-0f25-46f1-8937-8e4fbaac1d21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:00.506724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7712d6af-0f25-46f1-8937-8e4fbaac1d21 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:00.525566 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd09095-b4b1-479f-af3f-c42099a5261c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:00.528384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd09095-b4b1-479f-af3f-c42099a5261c admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:00.533769 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd09095-b4b1-479f-af3f-c42099a5261c admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:00.535724 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd09095-b4b1-479f-af3f-c42099a5261c admin admin] PATH_INFO: `/v3/projects/670af6cb988b46eeac8957e7404d7855/users/322a40c798b549b0a12cbbf44c98b631/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:00.593497 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd09095-b4b1-479f-af3f-c42099a5261c admin admin] RBAC: Authorizing `identity:create_grant(project_id=670af6cb988b46eeac8957e7404d7855, user_id=322a40c798b549b0a12cbbf44c98b631, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:00.595625 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 880/1781] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:03:00 2022] POST /identity/v3/projects => generated 401 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:00.597793 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712d6af-0f25-46f1-8937-8e4fbaac1d21 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:00.603524 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-7712d6af-0f25-46f1-8937-8e4fbaac1d21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:00.604625 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd09095-b4b1-479f-af3f-c42099a5261c admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:00.611426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:00.648625 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 902/1782] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:02:59 2022] PUT /identity/v3/projects/670af6cb988b46eeac8957e7404d7855/users/322a40c798b549b0a12cbbf44c98b631/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 717 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:00.651176 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24f135-7495-4666-ae82-4c0786d13ff6 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:00.652108 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24f135-7495-4666-ae82-4c0786d13ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:00.658066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24f135-7495-4666-ae82-4c0786d13ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:00.749223 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-4e24f135-7495-4666-ae82-4c0786d13ff6 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:00.750460 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-4e24f135-7495-4666-ae82-4c0786d13ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:00.776122 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712d6af-0f25-46f1-8937-8e4fbaac1d21 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:00.825191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:00.905531 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25e000-6f4b-4532-804a-b8bcda3461ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:00.906997 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25e000-6f4b-4532-804a-b8bcda3461ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:00.907440 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25e000-6f4b-4532-804a-b8bcda3461ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:00.909046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25e000-6f4b-4532-804a-b8bcda3461ab admin admin] PATH_INFO: `/v3/projects/4e11b728ee874e50aeee2fda90fe0c62` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:00.913406 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25e000-6f4b-4532-804a-b8bcda3461ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e11b728ee874e50aeee2fda90fe0c62)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:00.918416 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25e000-6f4b-4532-804a-b8bcda3461ab admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:00.937511 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 903/1783] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:00.939000 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42833b36-5ccf-4ede-bf2d-3ba40ba98f04 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:00.939210 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42833b36-5ccf-4ede-bf2d-3ba40ba98f04 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:00.939483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42833b36-5ccf-4ede-bf2d-3ba40ba98f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:01.020948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-42833b36-5ccf-4ede-bf2d-3ba40ba98f04 None None] MFA Rules not processed for user `b1c75368c00e4b0cb7b3163728843479`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:01.024507 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-42833b36-5ccf-4ede-bf2d-3ba40ba98f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:01.089178 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-6c25e000-6f4b-4532-804a-b8bcda3461ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e11b728ee874e50aeee2fda90fe0c62', 'request_id': 'req-6c25e000-6f4b-4532-804a-b8bcda3461ab'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:01.130722 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-6c25e000-6f4b-4532-804a-b8bcda3461ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e11b728ee874e50aeee2fda90fe0c62', 'request_id': 'req-6c25e000-6f4b-4532-804a-b8bcda3461ab'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:01.138693 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-6c25e000-6f4b-4532-804a-b8bcda3461ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e11b728ee874e50aeee2fda90fe0c62', 'request_id': 'req-6c25e000-6f4b-4532-804a-b8bcda3461ab'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:01.173069 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-6c25e000-6f4b-4532-804a-b8bcda3461ab admin admin] The token cache is being invalidate because project 4e11b728ee874e50aeee2fda90fe0c62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:03:01.173311 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-6c25e000-6f4b-4532-804a-b8bcda3461ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:01.177394 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-6c25e000-6f4b-4532-804a-b8bcda3461ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:01.183364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 881/1784] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:03:00 2022] DELETE /identity/v3/projects/4e11b728ee874e50aeee2fda90fe0c62 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:01.185649 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25e000-6f4b-4532-804a-b8bcda3461ab admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:01.189054 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-6c25e000-6f4b-4532-804a-b8bcda3461ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:01.238816 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 904/1785] 10.222.0.87 () {58 vars in 1015 bytes} [Fri Aug 5 20:03:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:01.240262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb2001-abbf-4766-9ba6-061c6b8bf097 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:01.240874 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb2001-abbf-4766-9ba6-061c6b8bf097 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:01.243146 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb2001-abbf-4766-9ba6-061c6b8bf097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:01.322113 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-48fb2001-abbf-4766-9ba6-061c6b8bf097 None None] MFA Rules not processed for user `322a40c798b549b0a12cbbf44c98b631`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:01.323713 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-48fb2001-abbf-4766-9ba6-061c6b8bf097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:01.369261 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25e000-6f4b-4532-804a-b8bcda3461ab admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:01.389423 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1545704-0167-4d22-86d7-db9f307cd77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:01.390603 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1545704-0167-4d22-86d7-db9f307cd77c admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:01.390896 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1545704-0167-4d22-86d7-db9f307cd77c admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:01.391221 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1545704-0167-4d22-86d7-db9f307cd77c admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:01.391911 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1545704-0167-4d22-86d7-db9f307cd77c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:01.393935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1545704-0167-4d22-86d7-db9f307cd77c admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:01.444033 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 882/1786] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:03:01 2022] POST /identity/v3/users => generated 301 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:01.448101 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03284b2-443c-4009-891d-9b1362a459d5 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:01.448397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03284b2-443c-4009-891d-9b1362a459d5 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:01.448692 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03284b2-443c-4009-891d-9b1362a459d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:01.491001 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-a03284b2-443c-4009-891d-9b1362a459d5 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:01.492238 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-a03284b2-443c-4009-891d-9b1362a459d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:01.493514 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 905/1787] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:01 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:01.494574 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb2001-abbf-4766-9ba6-061c6b8bf097 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:01.496116 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-48fb2001-abbf-4766-9ba6-061c6b8bf097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:01.512872 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 883/1788] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:01.512872 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03284b2-443c-4009-891d-9b1362a459d5 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:01.517993 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-a03284b2-443c-4009-891d-9b1362a459d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:01.522027 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb2001-abbf-4766-9ba6-061c6b8bf097 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:01.533993 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73795b-1091-445c-933a-c89e78bf91db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:01.536223 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73795b-1091-445c-933a-c89e78bf91db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:01.536876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73795b-1091-445c-933a-c89e78bf91db admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:01.537845 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73795b-1091-445c-933a-c89e78bf91db admin admin] PATH_INFO: `/v3/users/3569fb5aae9b4397b573020ab6bd2091` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:01.554427 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73795b-1091-445c-933a-c89e78bf91db admin admin] RBAC: Authorizing `identity:delete_user(user_id=3569fb5aae9b4397b573020ab6bd2091)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:01.558263 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73795b-1091-445c-933a-c89e78bf91db admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:01.644827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:01.689899 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:INSERT=1,keystone:SELECT=94,keystone:DELETE=5 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:01.701207 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03284b2-443c-4009-891d-9b1362a459d5 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:01.713101 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972bc35-04a9-4862-afb9-ab4e440d6369 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:01.715404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972bc35-04a9-4862-afb9-ab4e440d6369 service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:01.716612 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972bc35-04a9-4862-afb9-ab4e440d6369 service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:01.717104 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972bc35-04a9-4862-afb9-ab4e440d6369 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:01.730203 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4972bc35-04a9-4862-afb9-ab4e440d6369 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:01.731743 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4972bc35-04a9-4862-afb9-ab4e440d6369 service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:01.746728 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 884/1789] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:03:01.751973 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ef73795b-1091-445c-933a-c89e78bf91db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3569fb5aae9b4397b573020ab6bd2091'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:01.760315 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972bc35-04a9-4862-afb9-ab4e440d6369 service neutron] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:01.773745 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ef73795b-1091-445c-933a-c89e78bf91db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3569fb5aae9b4397b573020ab6bd2091'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:01.789516 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972bc35-04a9-4862-afb9-ab4e440d6369 service neutron] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:01.791859 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ef73795b-1091-445c-933a-c89e78bf91db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3569fb5aae9b4397b573020ab6bd2091'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:01.801172 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165882e-4581-48ce-8705-e4c5f37c773f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:01.802365 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165882e-4581-48ce-8705-e4c5f37c773f admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:01.802365 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165882e-4581-48ce-8705-e4c5f37c773f admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:01.802994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165882e-4581-48ce-8705-e4c5f37c773f admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:01.805901 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f165882e-4581-48ce-8705-e4c5f37c773f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:01.807684 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f165882e-4581-48ce-8705-e4c5f37c773f admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:01.837298 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 885/1790] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:01.837298 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165882e-4581-48ce-8705-e4c5f37c773f admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:01.838726 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-f165882e-4581-48ce-8705-e4c5f37c773f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:01.847548 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ef73795b-1091-445c-933a-c89e78bf91db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3569fb5aae9b4397b573020ab6bd2091'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:01.850896 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ef73795b-1091-445c-933a-c89e78bf91db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3569fb5aae9b4397b573020ab6bd2091'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:01.871749 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165882e-4581-48ce-8705-e4c5f37c773f admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:01.876945 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 906/1791] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:03:01 2022] DELETE /identity/v3/users/3569fb5aae9b4397b573020ab6bd2091 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:01.881221 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73795b-1091-445c-933a-c89e78bf91db admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:01.886901 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-ef73795b-1091-445c-933a-c89e78bf91db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:02.017349 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac1756-fadc-414e-8430-2884f4717d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:02.018079 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac1756-fadc-414e-8430-2884f4717d48 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:02.018438 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac1756-fadc-414e-8430-2884f4717d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:02.018789 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac1756-fadc-414e-8430-2884f4717d48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:02.021605 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dac1756-fadc-414e-8430-2884f4717d48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:02.023827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dac1756-fadc-414e-8430-2884f4717d48 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:02.098868 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 886/1792] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:03:01 2022] POST /identity/v3/projects => generated 405 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:02.104858 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac1756-fadc-414e-8430-2884f4717d48 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:02.126031 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73795b-1091-445c-933a-c89e78bf91db admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:02.129165 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-7dac1756-fadc-414e-8430-2884f4717d48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:02.321078 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac1756-fadc-414e-8430-2884f4717d48 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:02.335372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a69ae-63c4-479e-964b-7c007192915e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:02.336205 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a69ae-63c4-479e-964b-7c007192915e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:02.336619 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a69ae-63c4-479e-964b-7c007192915e admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:02.337062 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a69ae-63c4-479e-964b-7c007192915e admin admin] PATH_INFO: `/v3/projects/3b715e90c54141d4b78fad85302d6985` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:02.340287 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a69ae-63c4-479e-964b-7c007192915e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b715e90c54141d4b78fad85302d6985)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:02.344119 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a69ae-63c4-479e-964b-7c007192915e admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:02.432265 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52a354-5676-4f07-aa15-e01b143f5d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:02.437461 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52a354-5676-4f07-aa15-e01b143f5d55 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:02.438180 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52a354-5676-4f07-aa15-e01b143f5d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:02.438606 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52a354-5676-4f07-aa15-e01b143f5d55 admin admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:02.439967 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52a354-5676-4f07-aa15-e01b143f5d55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:03:02.441587 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52a354-5676-4f07-aa15-e01b143f5d55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:02.445740 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52a354-5676-4f07-aa15-e01b143f5d55 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:02.487311 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-4a6a69ae-63c4-479e-964b-7c007192915e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b715e90c54141d4b78fad85302d6985', 'request_id': 'req-4a6a69ae-63c4-479e-964b-7c007192915e'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:02.488152 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 907/1793] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 610 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:03:02.492822 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52a354-5676-4f07-aa15-e01b143f5d55 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:02.497164 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-de52a354-5676-4f07-aa15-e01b143f5d55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:02.504733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-4a6a69ae-63c4-479e-964b-7c007192915e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b715e90c54141d4b78fad85302d6985', 'request_id': 'req-4a6a69ae-63c4-479e-964b-7c007192915e'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:02.517318 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52a354-5676-4f07-aa15-e01b143f5d55 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:02.520547 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:02.529681 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-4a6a69ae-63c4-479e-964b-7c007192915e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b715e90c54141d4b78fad85302d6985', 'request_id': 'req-4a6a69ae-63c4-479e-964b-7c007192915e'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:02.575226 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-4a6a69ae-63c4-479e-964b-7c007192915e admin admin] The token cache is being invalidate because project 3b715e90c54141d4b78fad85302d6985 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:03:02.575708 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-4a6a69ae-63c4-479e-964b-7c007192915e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:02.576991 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-4a6a69ae-63c4-479e-964b-7c007192915e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:02.585113 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 887/1794] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:03:02 2022] DELETE /identity/v3/projects/3b715e90c54141d4b78fad85302d6985 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:02.587153 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a69ae-63c4-479e-964b-7c007192915e admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:02.592175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-4a6a69ae-63c4-479e-964b-7c007192915e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:02.636187 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:02.673031 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf279fe1-c633-48bd-87f4-80d791f1936e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:02.675101 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf279fe1-c633-48bd-87f4-80d791f1936e admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:02.682173 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf279fe1-c633-48bd-87f4-80d791f1936e admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:02.684978 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf279fe1-c633-48bd-87f4-80d791f1936e admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:02.684978 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf279fe1-c633-48bd-87f4-80d791f1936e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:02.687439 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf279fe1-c633-48bd-87f4-80d791f1936e admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:02.716693 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 908/1795] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:02.716693 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf279fe1-c633-48bd-87f4-80d791f1936e admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:02.730687 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-cf279fe1-c633-48bd-87f4-80d791f1936e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:02.779595 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf279fe1-c633-48bd-87f4-80d791f1936e admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:02.789294 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a5132-3143-4657-81dd-430505659317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:02.789942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a5132-3143-4657-81dd-430505659317 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:02.790269 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a5132-3143-4657-81dd-430505659317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:02.790642 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a5132-3143-4657-81dd-430505659317 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:02.792998 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a69ae-63c4-479e-964b-7c007192915e admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:02.793603 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a5132-3143-4657-81dd-430505659317 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:02.798438 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a5132-3143-4657-81dd-430505659317 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:02.812346 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01f084-96f4-4f80-95b8-4e2d09c0074e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:02.813030 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01f084-96f4-4f80-95b8-4e2d09c0074e admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:02.813310 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01f084-96f4-4f80-95b8-4e2d09c0074e admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:02.813848 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01f084-96f4-4f80-95b8-4e2d09c0074e admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:02.814526 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b01f084-96f4-4f80-95b8-4e2d09c0074e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:02.818086 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b01f084-96f4-4f80-95b8-4e2d09c0074e admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:02.850314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 909/1796] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:03:02 2022] POST /identity/v3/projects => generated 367 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:02.852190 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd68a8-1ce7-42b6-9e49-322a9e83d605 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:02.852835 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd68a8-1ce7-42b6-9e49-322a9e83d605 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:02.853262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd68a8-1ce7-42b6-9e49-322a9e83d605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:02.898537 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 888/1797] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:03:02 2022] POST /identity/v3/users => generated 309 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:02.902425 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01f084-96f4-4f80-95b8-4e2d09c0074e admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:02.907071 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-5b01f084-96f4-4f80-95b8-4e2d09c0074e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:02.925291 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-37fd68a8-1ce7-42b6-9e49-322a9e83d605 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:02.930542 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-37fd68a8-1ce7-42b6-9e49-322a9e83d605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:02.936008 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01f084-96f4-4f80-95b8-4e2d09c0074e admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:03.026694 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 910/1798] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:03.026694 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd68a8-1ce7-42b6-9e49-322a9e83d605 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:03.045688 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd68a8-1ce7-42b6-9e49-322a9e83d605 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:03.054580 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955782d-b7f4-4c07-b387-b1818aa0aae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:03.055883 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955782d-b7f4-4c07-b387-b1818aa0aae9 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:03.056424 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955782d-b7f4-4c07-b387-b1818aa0aae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:03.057021 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955782d-b7f4-4c07-b387-b1818aa0aae9 admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:03.058165 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0955782d-b7f4-4c07-b387-b1818aa0aae9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:03.061005 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0955782d-b7f4-4c07-b387-b1818aa0aae9 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:03.095313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 911/1799] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:03:03 2022] POST /identity/v3/users => generated 284 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:03.097518 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955782d-b7f4-4c07-b387-b1818aa0aae9 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:03.123447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955782d-b7f4-4c07-b387-b1818aa0aae9 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:03.130588 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5f1a7-f504-4ed4-b95a-6d480ae48e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:03.132015 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5f1a7-f504-4ed4-b95a-6d480ae48e98 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:03.132496 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5f1a7-f504-4ed4-b95a-6d480ae48e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:03.133027 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5f1a7-f504-4ed4-b95a-6d480ae48e98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:03.134468 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e5f1a7-f504-4ed4-b95a-6d480ae48e98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:03.137418 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e5f1a7-f504-4ed4-b95a-6d480ae48e98 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:03.137947 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a900e-f543-4860-9a21-f5172f100699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:03.138810 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a900e-f543-4860-9a21-f5172f100699 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:03.139099 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a900e-f543-4860-9a21-f5172f100699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:03.139385 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a900e-f543-4860-9a21-f5172f100699 admin admin] PATH_INFO: `/v3/projects/ac6fab5571ae4705827a24f15ba1b21e/users/886a28c0adae435998242d5a4053e2c4/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:03.169377 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 912/1800] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:03.170678 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5f1a7-f504-4ed4-b95a-6d480ae48e98 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:03.174812 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-17e5f1a7-f504-4ed4-b95a-6d480ae48e98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:03.198343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5f1a7-f504-4ed4-b95a-6d480ae48e98 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:03.208953 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27283c-da6c-4991-8e39-f81cb8f60813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:03.209523 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2a900e-f543-4860-9a21-f5172f100699 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac6fab5571ae4705827a24f15ba1b21e, user_id=886a28c0adae435998242d5a4053e2c4, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:03.209920 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27283c-da6c-4991-8e39-f81cb8f60813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:03.210339 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27283c-da6c-4991-8e39-f81cb8f60813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:03.210725 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27283c-da6c-4991-8e39-f81cb8f60813 admin admin] PATH_INFO: `/v3/projects/f4c29967ff11466f8f8780d067b24824` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:03.212196 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2a900e-f543-4860-9a21-f5172f100699 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:03.212949 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f27283c-da6c-4991-8e39-f81cb8f60813 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4c29967ff11466f8f8780d067b24824)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:03.215251 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f27283c-da6c-4991-8e39-f81cb8f60813 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:03.252244 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 889/1801] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:02 2022] PUT /identity/v3/projects/ac6fab5571ae4705827a24f15ba1b21e/users/886a28c0adae435998242d5a4053e2c4/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:03.260938 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e714f97-fbe7-40b0-8ff0-bb2244741ec2 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:03.261281 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e714f97-fbe7-40b0-8ff0-bb2244741ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:03.261645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e714f97-fbe7-40b0-8ff0-bb2244741ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:03.303260 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-7e714f97-fbe7-40b0-8ff0-bb2244741ec2 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:03.304668 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-7e714f97-fbe7-40b0-8ff0-bb2244741ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:03.328704 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-0f27283c-da6c-4991-8e39-f81cb8f60813 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4c29967ff11466f8f8780d067b24824', 'request_id': 'req-0f27283c-da6c-4991-8e39-f81cb8f60813'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:03.352975 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-0f27283c-da6c-4991-8e39-f81cb8f60813 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4c29967ff11466f8f8780d067b24824', 'request_id': 'req-0f27283c-da6c-4991-8e39-f81cb8f60813'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:03.379933 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-0f27283c-da6c-4991-8e39-f81cb8f60813 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4c29967ff11466f8f8780d067b24824', 'request_id': 'req-0f27283c-da6c-4991-8e39-f81cb8f60813'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:03.463212 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-0f27283c-da6c-4991-8e39-f81cb8f60813 admin admin] The token cache is being invalidate because project f4c29967ff11466f8f8780d067b24824 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:03:03.464081 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-0f27283c-da6c-4991-8e39-f81cb8f60813 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:03.469511 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-0f27283c-da6c-4991-8e39-f81cb8f60813 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:03.479761 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 913/1802] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:03:03 2022] DELETE /identity/v3/projects/f4c29967ff11466f8f8780d067b24824 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:03.482002 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27283c-da6c-4991-8e39-f81cb8f60813 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:03.483900 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 890/1803] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:03.484950 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e714f97-fbe7-40b0-8ff0-bb2244741ec2 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:03.488613 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-7e714f97-fbe7-40b0-8ff0-bb2244741ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:03.495608 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-0f27283c-da6c-4991-8e39-f81cb8f60813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:03.506871 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:03.518641 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e714f97-fbe7-40b0-8ff0-bb2244741ec2 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:03.531251 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a9b48-ac5b-4f94-8736-b5ca6f8f861d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:03.532088 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a9b48-ac5b-4f94-8736-b5ca6f8f861d admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:03.532371 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a9b48-ac5b-4f94-8736-b5ca6f8f861d admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:03.532671 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a9b48-ac5b-4f94-8736-b5ca6f8f861d admin admin] PATH_INFO: `/v3/projects/63a69467039e43e29d1a684b31ee814f/users/8fcc337258bb4efd8ed59f33da1e3485/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:03.557865 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27283c-da6c-4991-8e39-f81cb8f60813 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:03.572725 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e94995-e215-42f3-97a0-587e81dfe378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:03.573343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e94995-e215-42f3-97a0-587e81dfe378 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:03.573622 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e94995-e215-42f3-97a0-587e81dfe378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:03.573895 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e94995-e215-42f3-97a0-587e81dfe378 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:03.574974 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e94995-e215-42f3-97a0-587e81dfe378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:03.576526 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e94995-e215-42f3-97a0-587e81dfe378 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:03.580380 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a9b48-ac5b-4f94-8736-b5ca6f8f861d admin admin] RBAC: Authorizing `identity:create_grant(project_id=63a69467039e43e29d1a684b31ee814f, user_id=8fcc337258bb4efd8ed59f33da1e3485, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:03.583460 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a9b48-ac5b-4f94-8736-b5ca6f8f861d admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:03.621260 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 914/1804] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:03.624373 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57d4e7-6ba1-4430-a793-b81adf9d2a42 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:03.624672 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57d4e7-6ba1-4430-a793-b81adf9d2a42 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:03.624948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57d4e7-6ba1-4430-a793-b81adf9d2a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:03.627933 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 891/1805] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:03 2022] PUT /identity/v3/projects/63a69467039e43e29d1a684b31ee814f/users/8fcc337258bb4efd8ed59f33da1e3485/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:03.631903 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e83d2-5a9a-4f1f-ae5d-7f1856c348c6 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:03.632159 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e83d2-5a9a-4f1f-ae5d-7f1856c348c6 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:03.632437 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e83d2-5a9a-4f1f-ae5d-7f1856c348c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:03.643141 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:03.691660 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-f55e83d2-5a9a-4f1f-ae5d-7f1856c348c6 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:03.695696 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-f55e83d2-5a9a-4f1f-ae5d-7f1856c348c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:03.705110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-5e57d4e7-6ba1-4430-a793-b81adf9d2a42 None None] MFA Rules not processed for user `886a28c0adae435998242d5a4053e2c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:03.706201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5e57d4e7-6ba1-4430-a793-b81adf9d2a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:03.868060 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 892/1806] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:03.869497 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e83d2-5a9a-4f1f-ae5d-7f1856c348c6 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:03.895332 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e83d2-5a9a-4f1f-ae5d-7f1856c348c6 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:03.909293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd2ed5d-bbdf-48e0-a3b6-154c1f486639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:03.912259 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd2ed5d-bbdf-48e0-a3b6-154c1f486639 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:03.913138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd2ed5d-bbdf-48e0-a3b6-154c1f486639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:03.915095 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd2ed5d-bbdf-48e0-a3b6-154c1f486639 admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:03.916172 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd2ed5d-bbdf-48e0-a3b6-154c1f486639 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:03:03.917210 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd2ed5d-bbdf-48e0-a3b6-154c1f486639 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:03.921836 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd2ed5d-bbdf-48e0-a3b6-154c1f486639 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:03.943047 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 915/1807] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:03 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:03.947295 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57d4e7-6ba1-4430-a793-b81adf9d2a42 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:03.963924 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 893/1808] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:03:03.966235 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd2ed5d-bbdf-48e0-a3b6-154c1f486639 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:03.973369 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57d4e7-6ba1-4430-a793-b81adf9d2a42 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:03.986530 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2271deeb-a4fd-4f27-bcd2-a259d5594a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:03.996400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271deeb-a4fd-4f27-bcd2-a259d5594a97 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:03.999614 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd2ed5d-bbdf-48e0-a3b6-154c1f486639 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:04.000327 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271deeb-a4fd-4f27-bcd2-a259d5594a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:04.001076 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271deeb-a4fd-4f27-bcd2-a259d5594a97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:04.002282 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2271deeb-a4fd-4f27-bcd2-a259d5594a97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:04.006437 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2271deeb-a4fd-4f27-bcd2-a259d5594a97 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:04.015603 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7721a029-b600-4c81-b5c3-702b0d2b4114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:04.016846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721a029-b600-4c81-b5c3-702b0d2b4114 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:04.018088 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721a029-b600-4c81-b5c3-702b0d2b4114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:04.019625 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721a029-b600-4c81-b5c3-702b0d2b4114 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:04.022539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7721a029-b600-4c81-b5c3-702b0d2b4114 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:04.026254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7721a029-b600-4c81-b5c3-702b0d2b4114 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:04.040668 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 916/1809] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:04.042917 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11500543-343d-4a23-b763-f8f3dec8c6fd None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:04.043593 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11500543-343d-4a23-b763-f8f3dec8c6fd None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:04.044087 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11500543-343d-4a23-b763-f8f3dec8c6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:04.073346 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 894/1810] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:04.075511 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90f6e2-4d61-4ee9-a3d5-af015007de77 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:04.078129 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90f6e2-4d61-4ee9-a3d5-af015007de77 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:04.080468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90f6e2-4d61-4ee9-a3d5-af015007de77 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:04.086970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 895/1811] 10.222.0.39 () {54 vars in 921 bytes} [Fri Aug 5 20:03:04 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 20:03:04.088471 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90f6e2-4d61-4ee9-a3d5-af015007de77 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:04.094888 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-ec90f6e2-4d61-4ee9-a3d5-af015007de77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:04.104796 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-11500543-343d-4a23-b763-f8f3dec8c6fd None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:04.108807 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-11500543-343d-4a23-b763-f8f3dec8c6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:04.135582 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 917/1812] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:04.137604 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11500543-343d-4a23-b763-f8f3dec8c6fd None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:04.140632 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90f6e2-4d61-4ee9-a3d5-af015007de77 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:04.152269 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad02130-cfcb-40f7-98b9-006d616c67c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:04.153376 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad02130-cfcb-40f7-98b9-006d616c67c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:04.153883 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad02130-cfcb-40f7-98b9-006d616c67c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:04.154395 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad02130-cfcb-40f7-98b9-006d616c67c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:04.155298 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad02130-cfcb-40f7-98b9-006d616c67c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:04.157520 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad02130-cfcb-40f7-98b9-006d616c67c6 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:04.166474 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11500543-343d-4a23-b763-f8f3dec8c6fd None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:04.181488 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c72bc2-c136-4624-a8e0-69020e0fdde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:04.182883 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c72bc2-c136-4624-a8e0-69020e0fdde3 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:04.183680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c72bc2-c136-4624-a8e0-69020e0fdde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:04.184212 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c72bc2-c136-4624-a8e0-69020e0fdde3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:04.186917 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c72bc2-c136-4624-a8e0-69020e0fdde3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:04.189225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c72bc2-c136-4624-a8e0-69020e0fdde3 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:04.236962 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 896/1813] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:03:04 2022] POST /identity/v3/projects => generated 401 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:04.236962 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad02130-cfcb-40f7-98b9-006d616c67c6 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:04.277378 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 918/1814] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:03:04 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:04.278835 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c72bc2-c136-4624-a8e0-69020e0fdde3 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:04.405115 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad02130-cfcb-40f7-98b9-006d616c67c6 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:04.453216 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c72bc2-c136-4624-a8e0-69020e0fdde3 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:04.461798 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423bd06a-267c-425b-873a-9aec9664c9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:04.462495 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bd06a-267c-425b-873a-9aec9664c9f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:04.462925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bd06a-267c-425b-873a-9aec9664c9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:04.463222 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bd06a-267c-425b-873a-9aec9664c9f0 admin admin] PATH_INFO: `/v3/projects/63a69467039e43e29d1a684b31ee814f/users/8fcc337258bb4efd8ed59f33da1e3485/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:04.465997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423bd06a-267c-425b-873a-9aec9664c9f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63a69467039e43e29d1a684b31ee814f, user_id=8fcc337258bb4efd8ed59f33da1e3485, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:04.469461 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423bd06a-267c-425b-873a-9aec9664c9f0 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:04.490270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 919/1815] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:04 2022] PUT /identity/v3/projects/63a69467039e43e29d1a684b31ee814f/users/8fcc337258bb4efd8ed59f33da1e3485/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:04.500393 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161f3c1-5a41-442f-88ff-793da3ad2719 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:04.500393 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161f3c1-5a41-442f-88ff-793da3ad2719 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:04.500393 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161f3c1-5a41-442f-88ff-793da3ad2719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:04.522557 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:04.545126 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-e161f3c1-5a41-442f-88ff-793da3ad2719 None None] MFA Rules not processed for user `95e9f085e4f04d118373b8abff27543a`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:04.546797 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e161f3c1-5a41-442f-88ff-793da3ad2719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:04.580140 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3aae1-d2e7-4097-bd92-9275092e373a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:04.583530 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3aae1-d2e7-4097-bd92-9275092e373a admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:04.584003 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3aae1-d2e7-4097-bd92-9275092e373a admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:04.588002 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3aae1-d2e7-4097-bd92-9275092e373a admin admin] PATH_INFO: `/v3/projects/0525078eee2246d8826bd746fb4c2d72/users/ffb5b3a035b2463cb1f5bc055c82b09b/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:04.608069 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:04.631288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f3aae1-d2e7-4097-bd92-9275092e373a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0525078eee2246d8826bd746fb4c2d72, user_id=ffb5b3a035b2463cb1f5bc055c82b09b, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:04.635676 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f3aae1-d2e7-4097-bd92-9275092e373a admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:04.655856 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 920/1816] 10.222.0.39 () {58 vars in 1018 bytes} [Fri Aug 5 20:03:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:04.658386 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161f3c1-5a41-442f-88ff-793da3ad2719 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:04.667306 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 897/1817] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:04 2022] PUT /identity/v3/projects/0525078eee2246d8826bd746fb4c2d72/users/ffb5b3a035b2463cb1f5bc055c82b09b/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:04.669654 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3aae1-d2e7-4097-bd92-9275092e373a admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:04.775707 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3aae1-d2e7-4097-bd92-9275092e373a admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:04.779277 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161f3c1-5a41-442f-88ff-793da3ad2719 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:04.787203 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f9638-7a7f-42a2-8f92-b150559dd87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:04.790923 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334fec6-c058-4be4-b37f-4f5c258e823f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:04.791263 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f9638-7a7f-42a2-8f92-b150559dd87a admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:04.791557 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334fec6-c058-4be4-b37f-4f5c258e823f admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:04.791840 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334fec6-c058-4be4-b37f-4f5c258e823f admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:04.792108 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334fec6-c058-4be4-b37f-4f5c258e823f admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:04.792623 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f9638-7a7f-42a2-8f92-b150559dd87a admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:04.792811 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0334fec6-c058-4be4-b37f-4f5c258e823f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:04.793685 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f9638-7a7f-42a2-8f92-b150559dd87a admin admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:04.794720 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0334fec6-c058-4be4-b37f-4f5c258e823f admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:04.795841 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f9638-7a7f-42a2-8f92-b150559dd87a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:03:04.796422 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f9638-7a7f-42a2-8f92-b150559dd87a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:04.798868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f9638-7a7f-42a2-8f92-b150559dd87a admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:04.818270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 921/1818] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:03:04.819807 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f9638-7a7f-42a2-8f92-b150559dd87a admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:04.842674 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f9638-7a7f-42a2-8f92-b150559dd87a admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:04.849093 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 898/1819] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:03:04 2022] POST /identity/v3/users => generated 307 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:04.849309 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c4860-b3e6-46b4-b5ee-6bd7e148f08b None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:04.849556 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c4860-b3e6-46b4-b5ee-6bd7e148f08b None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:04.849973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c4860-b3e6-46b4-b5ee-6bd7e148f08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:04.858780 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c1735-70d5-401b-b98d-c21590105e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:04.867263 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c1735-70d5-401b-b98d-c21590105e5d admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:04.868337 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c1735-70d5-401b-b98d-c21590105e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:04.869299 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c1735-70d5-401b-b98d-c21590105e5d admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:04.871502 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c1735-70d5-401b-b98d-c21590105e5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:04.877211 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c1735-70d5-401b-b98d-c21590105e5d admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:04.909679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-579c4860-b3e6-46b4-b5ee-6bd7e148f08b None None] MFA Rules not processed for user `8fcc337258bb4efd8ed59f33da1e3485`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:04.912204 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-579c4860-b3e6-46b4-b5ee-6bd7e148f08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:04.938178 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 922/1820] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:03:04 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:04.945749 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f40b63-088b-43c3-83cd-404fa908ae4d None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:04.946671 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f40b63-088b-43c3-83cd-404fa908ae4d None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:04.947514 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f40b63-088b-43c3-83cd-404fa908ae4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:05.030805 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-e3f40b63-088b-43c3-83cd-404fa908ae4d None None] MFA Rules not processed for user `ffb5b3a035b2463cb1f5bc055c82b09b`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:05.033628 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e3f40b63-088b-43c3-83cd-404fa908ae4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:05.136802 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 899/1821] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:04 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:05.137724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c4860-b3e6-46b4-b5ee-6bd7e148f08b None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:05.169837 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c4860-b3e6-46b4-b5ee-6bd7e148f08b None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:05.179547 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb5e131-cc38-4547-8d2e-c6d9234a0334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:05.182826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5e131-cc38-4547-8d2e-c6d9234a0334 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:05.183385 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5e131-cc38-4547-8d2e-c6d9234a0334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:05.183954 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5e131-cc38-4547-8d2e-c6d9234a0334 admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:05.185417 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb5e131-cc38-4547-8d2e-c6d9234a0334 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:05.189294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb5e131-cc38-4547-8d2e-c6d9234a0334 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:05.198982 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 923/1822] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:05.198982 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f40b63-088b-43c3-83cd-404fa908ae4d None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:05.212167 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f40b63-088b-43c3-83cd-404fa908ae4d None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:05.218739 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf6f641-7e9c-4212-8f47-6e7cc613a127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:05.219446 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6f641-7e9c-4212-8f47-6e7cc613a127 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:05.219651 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6f641-7e9c-4212-8f47-6e7cc613a127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:05.219961 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6f641-7e9c-4212-8f47-6e7cc613a127 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:05.220697 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf6f641-7e9c-4212-8f47-6e7cc613a127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:05.223174 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf6f641-7e9c-4212-8f47-6e7cc613a127 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:05.244629 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 924/1823] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:05.244629 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf6f641-7e9c-4212-8f47-6e7cc613a127 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:05.246522 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 900/1824] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:03:05 2022] POST /identity/v3/projects => generated 367 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:05.247638 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb5e131-cc38-4547-8d2e-c6d9234a0334 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:05.251549 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-dcb5e131-cc38-4547-8d2e-c6d9234a0334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:05.260815 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf6f641-7e9c-4212-8f47-6e7cc613a127 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:05.298767 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:05.337289 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:05.389061 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a108273-c880-44ce-ae0f-3e9e9fdba19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:05.389743 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a108273-c880-44ce-ae0f-3e9e9fdba19f admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:05.390021 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a108273-c880-44ce-ae0f-3e9e9fdba19f admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:05.390294 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a108273-c880-44ce-ae0f-3e9e9fdba19f admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:05.390992 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a108273-c880-44ce-ae0f-3e9e9fdba19f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:05.398214 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a108273-c880-44ce-ae0f-3e9e9fdba19f admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:05.432927 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 925/1825] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:05.433866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a108273-c880-44ce-ae0f-3e9e9fdba19f admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:05.443426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb5e131-cc38-4547-8d2e-c6d9234a0334 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:05.451188 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f158f-4b24-4bf4-9c3a-4ed6953197b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:05.451810 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f158f-4b24-4bf4-9c3a-4ed6953197b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:05.452092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f158f-4b24-4bf4-9c3a-4ed6953197b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:05.452365 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f158f-4b24-4bf4-9c3a-4ed6953197b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:05.455143 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a108273-c880-44ce-ae0f-3e9e9fdba19f admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:05.464033 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55cf366-d0d4-45b8-bfea-70cd49f7e93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:05.464699 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55cf366-d0d4-45b8-bfea-70cd49f7e93e admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:05.464986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55cf366-d0d4-45b8-bfea-70cd49f7e93e admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:05.465265 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55cf366-d0d4-45b8-bfea-70cd49f7e93e admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:05.465951 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55cf366-d0d4-45b8-bfea-70cd49f7e93e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:05.467896 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55cf366-d0d4-45b8-bfea-70cd49f7e93e admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:05.515778 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 926/1826] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:03:05 2022] POST /identity/v3/projects => generated 367 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:05.516752 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55cf366-d0d4-45b8-bfea-70cd49f7e93e admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:05.629306 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f158f-4b24-4bf4-9c3a-4ed6953197b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:05.631430 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f158f-4b24-4bf4-9c3a-4ed6953197b3 service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:05.646527 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 901/1827] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:05 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:03:05.648010 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f158f-4b24-4bf4-9c3a-4ed6953197b3 service cinder] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:05.662071 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f158f-4b24-4bf4-9c3a-4ed6953197b3 service cinder] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:05.760042 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55cf366-d0d4-45b8-bfea-70cd49f7e93e admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:05.781064 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5253da-dba4-4a30-971d-ad792df11209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:05.788316 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5253da-dba4-4a30-971d-ad792df11209 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:05.792014 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5253da-dba4-4a30-971d-ad792df11209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:05.792338 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5253da-dba4-4a30-971d-ad792df11209 admin admin] PATH_INFO: `/v3/projects/5fab7c5df62b4a61bc09e491b27ae713/users/66546a7cf2ec49de9a24ae6953d40559/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:05.819613 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398774e-a7c9-4df0-8e2e-f146ed905ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:05.820671 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398774e-a7c9-4df0-8e2e-f146ed905ae1 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:05.821127 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398774e-a7c9-4df0-8e2e-f146ed905ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:05.821660 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398774e-a7c9-4df0-8e2e-f146ed905ae1 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:05.822497 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2398774e-a7c9-4df0-8e2e-f146ed905ae1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:05.824669 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2398774e-a7c9-4df0-8e2e-f146ed905ae1 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:05.868884 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5253da-dba4-4a30-971d-ad792df11209 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fab7c5df62b4a61bc09e491b27ae713, user_id=66546a7cf2ec49de9a24ae6953d40559, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:05.876438 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5253da-dba4-4a30-971d-ad792df11209 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:05.881019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 902/1828] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:03:05 2022] POST /identity/v3/users => generated 284 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:05.881019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398774e-a7c9-4df0-8e2e-f146ed905ae1 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:05.908450 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398774e-a7c9-4df0-8e2e-f146ed905ae1 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:05.947258 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 927/1829] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:05 2022] PUT /identity/v3/projects/5fab7c5df62b4a61bc09e491b27ae713/users/66546a7cf2ec49de9a24ae6953d40559/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:05.952875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5253da-dba4-4a30-971d-ad792df11209 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:05.963758 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-2e5253da-dba4-4a30-971d-ad792df11209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:06.027875 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:06.044540 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b03fb2-457b-4086-8997-82b02dd0a9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:06.045746 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b03fb2-457b-4086-8997-82b02dd0a9ba admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:06.046077 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b03fb2-457b-4086-8997-82b02dd0a9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:06.046359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b03fb2-457b-4086-8997-82b02dd0a9ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:06.047194 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b03fb2-457b-4086-8997-82b02dd0a9ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:06.067509 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b03fb2-457b-4086-8997-82b02dd0a9ba admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:06.096343 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 903/1830] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:06.097665 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b03fb2-457b-4086-8997-82b02dd0a9ba admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:06.124224 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5253da-dba4-4a30-971d-ad792df11209 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:06.125602 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b03fb2-457b-4086-8997-82b02dd0a9ba admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:06.143999 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1aaea-f15a-46ce-bd1b-296231657af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:06.144959 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1aaea-f15a-46ce-bd1b-296231657af5 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:06.145412 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1aaea-f15a-46ce-bd1b-296231657af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:06.147971 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1aaea-f15a-46ce-bd1b-296231657af5 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:06.148801 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba1aaea-f15a-46ce-bd1b-296231657af5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:06.150979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba1aaea-f15a-46ce-bd1b-296231657af5 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:06.157747 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e865fe-2f1d-4d3f-9fe0-c471c55560d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:06.158470 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e865fe-2f1d-4d3f-9fe0-c471c55560d9 service nova] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:06.158791 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e865fe-2f1d-4d3f-9fe0-c471c55560d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:06.159129 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e865fe-2f1d-4d3f-9fe0-c471c55560d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:06.197719 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e865fe-2f1d-4d3f-9fe0-c471c55560d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:06.199099 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 904/1831] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:03:06 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:06.199985 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1aaea-f15a-46ce-bd1b-296231657af5 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:06.202481 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e865fe-2f1d-4d3f-9fe0-c471c55560d9 service nova] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:06.222201 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1aaea-f15a-46ce-bd1b-296231657af5 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:06.229460 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7368074-a084-45c3-9e13-25ad456cb191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:06.230623 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7368074-a084-45c3-9e13-25ad456cb191 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:06.230623 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7368074-a084-45c3-9e13-25ad456cb191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:06.231223 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7368074-a084-45c3-9e13-25ad456cb191 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:06.232551 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7368074-a084-45c3-9e13-25ad456cb191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:06.234521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7368074-a084-45c3-9e13-25ad456cb191 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:06.235190 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 928/1832] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:03:06.236080 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e865fe-2f1d-4d3f-9fe0-c471c55560d9 service nova] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:06.302047 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 905/1833] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:06.305931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7368074-a084-45c3-9e13-25ad456cb191 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:06.314591 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e865fe-2f1d-4d3f-9fe0-c471c55560d9 service nova] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:06.339847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7368074-a084-45c3-9e13-25ad456cb191 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:06.341349 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5840588-a4ff-4595-97f2-12d9eeb7ac7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:06.343485 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5840588-a4ff-4595-97f2-12d9eeb7ac7b admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:06.345860 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5840588-a4ff-4595-97f2-12d9eeb7ac7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:06.346420 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5840588-a4ff-4595-97f2-12d9eeb7ac7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:06.350162 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5840588-a4ff-4595-97f2-12d9eeb7ac7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:06.357049 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5840588-a4ff-4595-97f2-12d9eeb7ac7b admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:06.373042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723df9b9-791b-412b-92a5-8fa354398a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:06.373771 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723df9b9-791b-412b-92a5-8fa354398a75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:06.374031 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723df9b9-791b-412b-92a5-8fa354398a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:06.379520 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723df9b9-791b-412b-92a5-8fa354398a75 admin admin] PATH_INFO: `/v3/projects/31443f67c6b74defa673e3165414137e/users/fc5b733da70a468e8f4b0b87ad0f2c66/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:06.411598 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 929/1834] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:06.413853 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5840588-a4ff-4595-97f2-12d9eeb7ac7b admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:06.448596 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723df9b9-791b-412b-92a5-8fa354398a75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31443f67c6b74defa673e3165414137e, user_id=fc5b733da70a468e8f4b0b87ad0f2c66, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:06.449241 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5840588-a4ff-4595-97f2-12d9eeb7ac7b admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:06.451371 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723df9b9-791b-412b-92a5-8fa354398a75 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:06.460362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dfa00e-73d1-4f0c-92cf-e8edffff36c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:06.461809 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dfa00e-73d1-4f0c-92cf-e8edffff36c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:06.462308 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dfa00e-73d1-4f0c-92cf-e8edffff36c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:06.462787 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dfa00e-73d1-4f0c-92cf-e8edffff36c3 admin admin] PATH_INFO: `/v3/users/322a40c798b549b0a12cbbf44c98b631` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:06.469031 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dfa00e-73d1-4f0c-92cf-e8edffff36c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=322a40c798b549b0a12cbbf44c98b631)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:06.473844 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dfa00e-73d1-4f0c-92cf-e8edffff36c3 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:06.476837 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:06.501915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 906/1835] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:06 2022] PUT /identity/v3/projects/31443f67c6b74defa673e3165414137e/users/fc5b733da70a468e8f4b0b87ad0f2c66/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:06.503254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723df9b9-791b-412b-92a5-8fa354398a75 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:06.631285 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-b1dfa00e-73d1-4f0c-92cf-e8edffff36c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '322a40c798b549b0a12cbbf44c98b631'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:06.646185 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-b1dfa00e-73d1-4f0c-92cf-e8edffff36c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '322a40c798b549b0a12cbbf44c98b631'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:06.664873 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-b1dfa00e-73d1-4f0c-92cf-e8edffff36c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '322a40c798b549b0a12cbbf44c98b631'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:06.710882 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-b1dfa00e-73d1-4f0c-92cf-e8edffff36c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '322a40c798b549b0a12cbbf44c98b631'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:06.712100 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-b1dfa00e-73d1-4f0c-92cf-e8edffff36c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '322a40c798b549b0a12cbbf44c98b631'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:06.714864 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723df9b9-791b-412b-92a5-8fa354398a75 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:06.731610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8443fac9-d877-411e-b55a-f7e68ef63e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:06.732385 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443fac9-d877-411e-b55a-f7e68ef63e7e admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:06.732767 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 930/1836] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:03:06 2022] DELETE /identity/v3/users/322a40c798b549b0a12cbbf44c98b631 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:06.733924 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dfa00e-73d1-4f0c-92cf-e8edffff36c3 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:06.734694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443fac9-d877-411e-b55a-f7e68ef63e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:06.735028 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443fac9-d877-411e-b55a-f7e68ef63e7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:06.735710 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8443fac9-d877-411e-b55a-f7e68ef63e7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:06.737747 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8443fac9-d877-411e-b55a-f7e68ef63e7e admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:06.740349 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-b1dfa00e-73d1-4f0c-92cf-e8edffff36c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:06.768260 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 907/1837] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:06.776708 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8443fac9-d877-411e-b55a-f7e68ef63e7e admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:06.780677 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-8443fac9-d877-411e-b55a-f7e68ef63e7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:06.809545 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8443fac9-d877-411e-b55a-f7e68ef63e7e admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:06.900354 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dfa00e-73d1-4f0c-92cf-e8edffff36c3 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:06.930969 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fcd05f-fd8d-4eab-b39a-6c45920a7f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:06.932981 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcd05f-fd8d-4eab-b39a-6c45920a7f9f admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:06.933449 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcd05f-fd8d-4eab-b39a-6c45920a7f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:06.933566 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcd05f-fd8d-4eab-b39a-6c45920a7f9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:06.935832 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fcd05f-fd8d-4eab-b39a-6c45920a7f9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:06.939505 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fcd05f-fd8d-4eab-b39a-6c45920a7f9f admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:06.978779 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 931/1838] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:06.980200 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae28c1-2dde-4eac-9088-55970fd6284a None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:06.989905 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbaf966-0cba-49aa-8a06-575f99bc44fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:06.993523 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbaf966-0cba-49aa-8a06-575f99bc44fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:06.993650 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae28c1-2dde-4eac-9088-55970fd6284a None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:06.993923 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae28c1-2dde-4eac-9088-55970fd6284a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:06.994431 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbaf966-0cba-49aa-8a06-575f99bc44fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:07.000250 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbaf966-0cba-49aa-8a06-575f99bc44fa admin admin] PATH_INFO: `/v3/projects/5fab7c5df62b4a61bc09e491b27ae713/users/66546a7cf2ec49de9a24ae6953d40559/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:07.006760 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbaf966-0cba-49aa-8a06-575f99bc44fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fab7c5df62b4a61bc09e491b27ae713, user_id=66546a7cf2ec49de9a24ae6953d40559, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:07.010239 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbaf966-0cba-49aa-8a06-575f99bc44fa admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:07.052809 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 908/1839] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:06 2022] PUT /identity/v3/projects/5fab7c5df62b4a61bc09e491b27ae713/users/66546a7cf2ec49de9a24ae6953d40559/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:07.055336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e191524-350c-4306-86f9-c6fe4cadb66f None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:07.055960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e191524-350c-4306-86f9-c6fe4cadb66f None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:07.056551 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e191524-350c-4306-86f9-c6fe4cadb66f None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:07.058696 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 909/1840] 10.222.0.39 () {54 vars in 921 bytes} [Fri Aug 5 20:03:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 20:03:07.060400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d43d2-8ca8-4067-aeb2-48d1b6272e67 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:07.061025 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d43d2-8ca8-4067-aeb2-48d1b6272e67 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:07.061604 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d43d2-8ca8-4067-aeb2-48d1b6272e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:07.087852 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-000d43d2-8ca8-4067-aeb2-48d1b6272e67 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:07.089598 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-000d43d2-8ca8-4067-aeb2-48d1b6272e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:07.116837 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-c3ae28c1-2dde-4eac-9088-55970fd6284a None None] MFA Rules not processed for user `fc5b733da70a468e8f4b0b87ad0f2c66`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:07.121626 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-c3ae28c1-2dde-4eac-9088-55970fd6284a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:07.126760 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:07.192780 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 910/1841] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:07.193946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d43d2-8ca8-4067-aeb2-48d1b6272e67 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:07.195443 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-000d43d2-8ca8-4067-aeb2-48d1b6272e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:07.219914 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d43d2-8ca8-4067-aeb2-48d1b6272e67 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:07.233290 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e090331d-2506-4bed-845f-cd0495f9b812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:07.233990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090331d-2506-4bed-845f-cd0495f9b812 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:07.234290 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090331d-2506-4bed-845f-cd0495f9b812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:07.234570 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090331d-2506-4bed-845f-cd0495f9b812 admin admin] PATH_INFO: `/v3/projects/4d879df100d4411ba8e9e750380e6857/users/c8c4abac652f4542a08ea6e54c6a9206/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:07.261360 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e090331d-2506-4bed-845f-cd0495f9b812 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d879df100d4411ba8e9e750380e6857, user_id=c8c4abac652f4542a08ea6e54c6a9206, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:07.266156 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e090331d-2506-4bed-845f-cd0495f9b812 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:07.302031 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 932/1842] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:07.303027 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae28c1-2dde-4eac-9088-55970fd6284a None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:07.316840 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 911/1843] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:07 2022] PUT /identity/v3/projects/4d879df100d4411ba8e9e750380e6857/users/c8c4abac652f4542a08ea6e54c6a9206/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:07.317906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:07.319066 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecaa5b-a648-448b-af1c-1594d65aa4d6 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:07.319620 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecaa5b-a648-448b-af1c-1594d65aa4d6 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:07.320078 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecaa5b-a648-448b-af1c-1594d65aa4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:07.338669 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae28c1-2dde-4eac-9088-55970fd6284a None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:07.437335 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-3fecaa5b-a648-448b-af1c-1594d65aa4d6 None None] MFA Rules not processed for user `66546a7cf2ec49de9a24ae6953d40559`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:07.454076 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-3fecaa5b-a648-448b-af1c-1594d65aa4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:07.586709 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde4a19-2b6d-4469-b59d-ee89a789a4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:07.588495 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde4a19-2b6d-4469-b59d-ee89a789a4ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:07.589016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde4a19-2b6d-4469-b59d-ee89a789a4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:07.589501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde4a19-2b6d-4469-b59d-ee89a789a4ac admin admin] PATH_INFO: `/v3/projects/be9db04a0d554c45989492d020ec279f/users/04bc57f4bc9042628e177c80fbf1e4b1/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:07.616243 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde4a19-2b6d-4469-b59d-ee89a789a4ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=be9db04a0d554c45989492d020ec279f, user_id=04bc57f4bc9042628e177c80fbf1e4b1, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:07.630434 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde4a19-2b6d-4469-b59d-ee89a789a4ac admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:07.649514 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 933/1844] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:07 2022] PUT /identity/v3/projects/be9db04a0d554c45989492d020ec279f/users/04bc57f4bc9042628e177c80fbf1e4b1/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:07.655340 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d27d58-77b6-4f85-a008-d02f50da0bc9 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:07.655632 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d27d58-77b6-4f85-a008-d02f50da0bc9 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:07.655924 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d27d58-77b6-4f85-a008-d02f50da0bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:07.660943 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 912/1845] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:07 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:07.662211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fecaa5b-a648-448b-af1c-1594d65aa4d6 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:07.664170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-3fecaa5b-a648-448b-af1c-1594d65aa4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:07.684336 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-34d27d58-77b6-4f85-a008-d02f50da0bc9 None None] MFA Rules not processed for user `95e9f085e4f04d118373b8abff27543a`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:07.686073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-34d27d58-77b6-4f85-a008-d02f50da0bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:07.753687 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fecaa5b-a648-448b-af1c-1594d65aa4d6 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:07.833528 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 934/1846] 10.222.0.39 () {58 vars in 1018 bytes} [Fri Aug 5 20:03:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:07.841032 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d27d58-77b6-4f85-a008-d02f50da0bc9 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:07.845970 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-34d27d58-77b6-4f85-a008-d02f50da0bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:07.856974 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:07.869731 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a688623-f208-43e7-ba3a-6b421636dbe1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:07.870382 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a688623-f208-43e7-ba3a-6b421636dbe1 service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:07.871464 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a688623-f208-43e7-ba3a-6b421636dbe1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:07.891042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a688623-f208-43e7-ba3a-6b421636dbe1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:08.020388 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a688623-f208-43e7-ba3a-6b421636dbe1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:08.021828 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d27d58-77b6-4f85-a008-d02f50da0bc9 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:08.023333 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a688623-f208-43e7-ba3a-6b421636dbe1 service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:08.030897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f13314-5a9b-4cdb-b1c9-b18b859fdff6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:08.031825 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f13314-5a9b-4cdb-b1c9-b18b859fdff6 service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:08.032301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f13314-5a9b-4cdb-b1c9-b18b859fdff6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:08.032846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f13314-5a9b-4cdb-b1c9-b18b859fdff6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:08.061695 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 913/1847] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:03:08.066548 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a688623-f208-43e7-ba3a-6b421636dbe1 service neutron] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:08.078801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:08.100787 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a688623-f208-43e7-ba3a-6b421636dbe1 service neutron] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:08.126310 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a56c3f1-183d-4e1a-9f1a-5928601e3327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:08.127027 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56c3f1-183d-4e1a-9f1a-5928601e3327 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:08.127328 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56c3f1-183d-4e1a-9f1a-5928601e3327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:08.127713 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56c3f1-183d-4e1a-9f1a-5928601e3327 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:08.140805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a56c3f1-183d-4e1a-9f1a-5928601e3327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:08.142740 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a56c3f1-183d-4e1a-9f1a-5928601e3327 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:08.184273 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 914/1848] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:08.195310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f13314-5a9b-4cdb-b1c9-b18b859fdff6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:08.198599 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56faa34a-8d11-40bf-a809-52cd31e792c6 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:08.198941 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56faa34a-8d11-40bf-a809-52cd31e792c6 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:08.199228 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56faa34a-8d11-40bf-a809-52cd31e792c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:08.202487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f13314-5a9b-4cdb-b1c9-b18b859fdff6 service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:08.240731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 935/1849] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:03:08.245216 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f13314-5a9b-4cdb-b1c9-b18b859fdff6 service cinder] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:08.250057 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-45f13314-5a9b-4cdb-b1c9-b18b859fdff6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:08.290340 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f13314-5a9b-4cdb-b1c9-b18b859fdff6 service cinder] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:08.300784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb5110-e241-4eba-9b0a-a3734a095d03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:08.302119 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb5110-e241-4eba-9b0a-a3734a095d03 service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:08.303235 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb5110-e241-4eba-9b0a-a3734a095d03 service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:08.303537 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb5110-e241-4eba-9b0a-a3734a095d03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:08.340124 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-56faa34a-8d11-40bf-a809-52cd31e792c6 None None] MFA Rules not processed for user `04bc57f4bc9042628e177c80fbf1e4b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:08.342348 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-56faa34a-8d11-40bf-a809-52cd31e792c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:08.458227 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb5110-e241-4eba-9b0a-a3734a095d03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:08.460546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb5110-e241-4eba-9b0a-a3734a095d03 service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:08.479804 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 936/1850] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:08 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:03:08.481184 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb5110-e241-4eba-9b0a-a3734a095d03 service cinder] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:08.521671 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb5110-e241-4eba-9b0a-a3734a095d03 service cinder] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:08.532053 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201f57a-be35-4b22-a3bf-9329cb8bd06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:08.533756 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201f57a-be35-4b22-a3bf-9329cb8bd06e admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:08.535391 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201f57a-be35-4b22-a3bf-9329cb8bd06e admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:08.535834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201f57a-be35-4b22-a3bf-9329cb8bd06e admin admin] PATH_INFO: `/v3/projects/4d879df100d4411ba8e9e750380e6857/users/c8c4abac652f4542a08ea6e54c6a9206/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:08.542239 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5201f57a-be35-4b22-a3bf-9329cb8bd06e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d879df100d4411ba8e9e750380e6857, user_id=c8c4abac652f4542a08ea6e54c6a9206, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:08.552169 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5201f57a-be35-4b22-a3bf-9329cb8bd06e admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:08.606467 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 915/1851] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 422 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:08.612496 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 937/1852] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:08 2022] PUT /identity/v3/projects/4d879df100d4411ba8e9e750380e6857/users/c8c4abac652f4542a08ea6e54c6a9206/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:08.614906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201f57a-be35-4b22-a3bf-9329cb8bd06e admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:08.617898 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56faa34a-8d11-40bf-a809-52cd31e792c6 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:08.620400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5201f57a-be35-4b22-a3bf-9329cb8bd06e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:08.637433 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-56faa34a-8d11-40bf-a809-52cd31e792c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:08.715941 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56faa34a-8d11-40bf-a809-52cd31e792c6 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:08.864164 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201f57a-be35-4b22-a3bf-9329cb8bd06e admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:08.881199 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1045fd-8455-4cc3-9bcf-25cf1c25f1e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:08.882957 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1045fd-8455-4cc3-9bcf-25cf1c25f1e4 service nova] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:08.884262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1045fd-8455-4cc3-9bcf-25cf1c25f1e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:08.884895 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1045fd-8455-4cc3-9bcf-25cf1c25f1e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:08.899072 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91050463-f81f-442e-9ef8-5b7d4a6142be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:08.899747 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91050463-f81f-442e-9ef8-5b7d4a6142be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:08.900031 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91050463-f81f-442e-9ef8-5b7d4a6142be admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:08.901124 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91050463-f81f-442e-9ef8-5b7d4a6142be admin admin] PATH_INFO: `/v3/users/fc5b733da70a468e8f4b0b87ad0f2c66` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:08.903592 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91050463-f81f-442e-9ef8-5b7d4a6142be admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc5b733da70a468e8f4b0b87ad0f2c66)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:08.905676 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91050463-f81f-442e-9ef8-5b7d4a6142be admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:08.910533 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1045fd-8455-4cc3-9bcf-25cf1c25f1e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:08.914021 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1045fd-8455-4cc3-9bcf-25cf1c25f1e4 service nova] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:08.938940 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 938/1853] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:03:08.939819 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1045fd-8455-4cc3-9bcf-25cf1c25f1e4 service nova] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:08.943115 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:08.979602 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1045fd-8455-4cc3-9bcf-25cf1c25f1e4 service nova] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:09.006298 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9458f39b-9946-4d56-8c77-1262ceaccf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:09.014172 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458f39b-9946-4d56-8c77-1262ceaccf95 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:09.014530 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458f39b-9946-4d56-8c77-1262ceaccf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:09.014790 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458f39b-9946-4d56-8c77-1262ceaccf95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:09.015535 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9458f39b-9946-4d56-8c77-1262ceaccf95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:09.023521 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9458f39b-9946-4d56-8c77-1262ceaccf95 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:09.062314 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-91050463-f81f-442e-9ef8-5b7d4a6142be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc5b733da70a468e8f4b0b87ad0f2c66'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:09.085628 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-91050463-f81f-442e-9ef8-5b7d4a6142be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc5b733da70a468e8f4b0b87ad0f2c66'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:09.109394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 939/1854] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:03:08 2022] POST /identity/v3/projects => generated 367 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:09.121476 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a4315-f91d-4663-b800-c11f276c3d6f None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:09.121776 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a4315-f91d-4663-b800-c11f276c3d6f None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:09.122128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a4315-f91d-4663-b800-c11f276c3d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:09.132490 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-91050463-f81f-442e-9ef8-5b7d4a6142be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc5b733da70a468e8f4b0b87ad0f2c66'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:09.134912 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-91050463-f81f-442e-9ef8-5b7d4a6142be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc5b733da70a468e8f4b0b87ad0f2c66'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:09.164884 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-91050463-f81f-442e-9ef8-5b7d4a6142be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc5b733da70a468e8f4b0b87ad0f2c66'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:09.185905 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 916/1855] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:03:08 2022] DELETE /identity/v3/users/fc5b733da70a468e8f4b0b87ad0f2c66 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:09.185905 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91050463-f81f-442e-9ef8-5b7d4a6142be admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:09.188504 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-91050463-f81f-442e-9ef8-5b7d4a6142be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:09.210468 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-036a4315-f91d-4663-b800-c11f276c3d6f None None] MFA Rules not processed for user `c8c4abac652f4542a08ea6e54c6a9206`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:09.214252 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-036a4315-f91d-4663-b800-c11f276c3d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:09.234354 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:09.340090 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91050463-f81f-442e-9ef8-5b7d4a6142be admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:09.345033 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7753539-e95e-4827-9ca4-ae9939cc1af6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:09.345753 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7753539-e95e-4827-9ca4-ae9939cc1af6 service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:09.346371 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7753539-e95e-4827-9ca4-ae9939cc1af6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:09.346766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7753539-e95e-4827-9ca4-ae9939cc1af6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:09.349615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-e7753539-e95e-4827-9ca4-ae9939cc1af6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:09.486308 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 940/1856] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:09 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:09.488664 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a4315-f91d-4663-b800-c11f276c3d6f None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:09.491507 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-036a4315-f91d-4663-b800-c11f276c3d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:09.507469 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7753539-e95e-4827-9ca4-ae9939cc1af6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:09.515471 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7753539-e95e-4827-9ca4-ae9939cc1af6 service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:09.536089 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 917/1857] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:09 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:03:09.537255 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7753539-e95e-4827-9ca4-ae9939cc1af6 service cinder] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:09.538833 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-e7753539-e95e-4827-9ca4-ae9939cc1af6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:09.657955 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:09.691161 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a4315-f91d-4663-b800-c11f276c3d6f None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:09.696738 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7753539-e95e-4827-9ca4-ae9939cc1af6 service cinder] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:09.703295 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4bf31-0954-4023-8118-4b1c0b57669a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:09.705211 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4bf31-0954-4023-8118-4b1c0b57669a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:09.705504 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4bf31-0954-4023-8118-4b1c0b57669a admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:09.706470 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4bf31-0954-4023-8118-4b1c0b57669a admin admin] PATH_INFO: `/v3/projects/670af6cb988b46eeac8957e7404d7855` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:09.708804 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf13b4-3d9f-4906-b76b-644a82a0fc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:09.710832 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf13b4-3d9f-4906-b76b-644a82a0fc79 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:09.711037 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a4bf31-0954-4023-8118-4b1c0b57669a admin admin] RBAC: Authorizing `identity:delete_project(project_id=670af6cb988b46eeac8957e7404d7855)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:09.711205 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf13b4-3d9f-4906-b76b-644a82a0fc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:09.711383 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf13b4-3d9f-4906-b76b-644a82a0fc79 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:09.712053 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbf13b4-3d9f-4906-b76b-644a82a0fc79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:09.712955 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a4bf31-0954-4023-8118-4b1c0b57669a admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:09.714035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbf13b4-3d9f-4906-b76b-644a82a0fc79 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:09.777827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 918/1858] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:03:09 2022] POST /identity/v3/users => generated 290 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:09.777827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599e8fb-1eb1-443f-9299-591377be1a29 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:09.777827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599e8fb-1eb1-443f-9299-591377be1a29 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:09.779403 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599e8fb-1eb1-443f-9299-591377be1a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:09.825200 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-0599e8fb-1eb1-443f-9299-591377be1a29 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:09.826449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-0599e8fb-1eb1-443f-9299-591377be1a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:09.835479 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-70a4bf31-0954-4023-8118-4b1c0b57669a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '670af6cb988b46eeac8957e7404d7855', 'request_id': 'req-70a4bf31-0954-4023-8118-4b1c0b57669a'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:09.843348 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-70a4bf31-0954-4023-8118-4b1c0b57669a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '670af6cb988b46eeac8957e7404d7855', 'request_id': 'req-70a4bf31-0954-4023-8118-4b1c0b57669a'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:09.860004 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-70a4bf31-0954-4023-8118-4b1c0b57669a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '670af6cb988b46eeac8957e7404d7855', 'request_id': 'req-70a4bf31-0954-4023-8118-4b1c0b57669a'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:09.899658 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-70a4bf31-0954-4023-8118-4b1c0b57669a admin admin] The token cache is being invalidate because project 670af6cb988b46eeac8957e7404d7855 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:03:09.899931 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-70a4bf31-0954-4023-8118-4b1c0b57669a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:09.902758 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-70a4bf31-0954-4023-8118-4b1c0b57669a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:09.907470 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 941/1859] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:03:09 2022] DELETE /identity/v3/projects/670af6cb988b46eeac8957e7404d7855 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:09.911950 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4bf31-0954-4023-8118-4b1c0b57669a admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:09.917765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-70a4bf31-0954-4023-8118-4b1c0b57669a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:10.028269 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:10.089763 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 919/1860] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:10.090662 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4bf31-0954-4023-8118-4b1c0b57669a admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:10.091129 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599e8fb-1eb1-443f-9299-591377be1a29 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:10.092794 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-0599e8fb-1eb1-443f-9299-591377be1a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:10.111927 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599e8fb-1eb1-443f-9299-591377be1a29 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:10.127804 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868cddb5-6f03-4073-ad7d-5ef80227d8bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:10.128593 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868cddb5-6f03-4073-ad7d-5ef80227d8bc service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:10.129083 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868cddb5-6f03-4073-ad7d-5ef80227d8bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:10.129384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868cddb5-6f03-4073-ad7d-5ef80227d8bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:10.130097 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7f575b-d8e9-410e-895f-062f73327178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:10.131006 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f575b-d8e9-410e-895f-062f73327178 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:10.131488 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f575b-d8e9-410e-895f-062f73327178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:10.132282 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-868cddb5-6f03-4073-ad7d-5ef80227d8bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:10.137567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f575b-d8e9-410e-895f-062f73327178 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:10.140771 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7f575b-d8e9-410e-895f-062f73327178 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:10.144020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7f575b-d8e9-410e-895f-062f73327178 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:10.182689 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 920/1861] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:10.184556 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7f575b-d8e9-410e-895f-062f73327178 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:10.190504 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-6d7f575b-d8e9-410e-895f-062f73327178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:10.230849 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7f575b-d8e9-410e-895f-062f73327178 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:10.241605 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc926b11-5242-479e-aca4-e822f143a4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:10.242446 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc926b11-5242-479e-aca4-e822f143a4eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:10.242825 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc926b11-5242-479e-aca4-e822f143a4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:10.243195 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc926b11-5242-479e-aca4-e822f143a4eb admin admin] PATH_INFO: `/v3/users/8fcc337258bb4efd8ed59f33da1e3485` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:10.244484 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc926b11-5242-479e-aca4-e822f143a4eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fcc337258bb4efd8ed59f33da1e3485)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:10.246521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc926b11-5242-479e-aca4-e822f143a4eb admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:10.258250 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868cddb5-6f03-4073-ad7d-5ef80227d8bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:10.259655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868cddb5-6f03-4073-ad7d-5ef80227d8bc service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:10.277884 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 942/1862] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:09 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:03:10.278731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868cddb5-6f03-4073-ad7d-5ef80227d8bc service cinder] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:10.300769 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868cddb5-6f03-4073-ad7d-5ef80227d8bc service cinder] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:10.315064 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ab442-894a-436a-91e1-c551028531ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:10.316388 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ab442-894a-436a-91e1-c551028531ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:10.316826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ab442-894a-436a-91e1-c551028531ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:10.317496 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ab442-894a-436a-91e1-c551028531ae admin admin] PATH_INFO: `/v3/projects/19a89aa5894d4b0f8034d04bdc4494d0/users/d085c6d872e94543b2bfd3e19311bf20/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:10.388017 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651ab442-894a-436a-91e1-c551028531ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=19a89aa5894d4b0f8034d04bdc4494d0, user_id=d085c6d872e94543b2bfd3e19311bf20, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:10.391347 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651ab442-894a-436a-91e1-c551028531ae admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:10.426283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 943/1863] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:10 2022] PUT /identity/v3/projects/19a89aa5894d4b0f8034d04bdc4494d0/users/d085c6d872e94543b2bfd3e19311bf20/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:10.426283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ab442-894a-436a-91e1-c551028531ae admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:10.432534 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-dc926b11-5242-479e-aca4-e822f143a4eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fcc337258bb4efd8ed59f33da1e3485'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:10.440165 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-651ab442-894a-436a-91e1-c551028531ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:10.455472 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-dc926b11-5242-479e-aca4-e822f143a4eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fcc337258bb4efd8ed59f33da1e3485'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:10.501388 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-dc926b11-5242-479e-aca4-e822f143a4eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fcc337258bb4efd8ed59f33da1e3485'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:10.504787 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-dc926b11-5242-479e-aca4-e822f143a4eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fcc337258bb4efd8ed59f33da1e3485'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:10.542375 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-dc926b11-5242-479e-aca4-e822f143a4eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fcc337258bb4efd8ed59f33da1e3485'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:10.553684 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 921/1864] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:03:10 2022] DELETE /identity/v3/users/8fcc337258bb4efd8ed59f33da1e3485 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:10.555567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc926b11-5242-479e-aca4-e822f143a4eb admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:10.558707 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-dc926b11-5242-479e-aca4-e822f143a4eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:10.650711 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ab442-894a-436a-91e1-c551028531ae admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:10.673927 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:10.724056 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc926b11-5242-479e-aca4-e822f143a4eb admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:10.731261 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6f50d-1f59-4fa3-bba4-42d17032f682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:10.732092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6f50d-1f59-4fa3-bba4-42d17032f682 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:10.732800 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6f50d-1f59-4fa3-bba4-42d17032f682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:10.733349 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6f50d-1f59-4fa3-bba4-42d17032f682 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:10.734376 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc6f50d-1f59-4fa3-bba4-42d17032f682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:10.739417 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc6f50d-1f59-4fa3-bba4-42d17032f682 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:10.762906 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 922/1865] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:10.766967 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6f50d-1f59-4fa3-bba4-42d17032f682 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:10.784026 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6f50d-1f59-4fa3-bba4-42d17032f682 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:10.791573 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce579b2e-e8b2-4ec1-a93d-64087e621995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:10.791573 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce579b2e-e8b2-4ec1-a93d-64087e621995 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:10.792089 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce579b2e-e8b2-4ec1-a93d-64087e621995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:10.792473 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce579b2e-e8b2-4ec1-a93d-64087e621995 admin admin] PATH_INFO: `/v3/projects/19a89aa5894d4b0f8034d04bdc4494d0/users/d085c6d872e94543b2bfd3e19311bf20/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:10.794826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce579b2e-e8b2-4ec1-a93d-64087e621995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19a89aa5894d4b0f8034d04bdc4494d0, user_id=d085c6d872e94543b2bfd3e19311bf20, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:10.798355 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce579b2e-e8b2-4ec1-a93d-64087e621995 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:10.805381 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4aea2-1c0d-4e3f-b1c0-24e742824cbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:10.806555 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4aea2-1c0d-4e3f-b1c0-24e742824cbe service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:10.807208 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4aea2-1c0d-4e3f-b1c0-24e742824cbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:10.807560 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4aea2-1c0d-4e3f-b1c0-24e742824cbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:10.810069 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-74e4aea2-1c0d-4e3f-b1c0-24e742824cbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:10.823557 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 923/1866] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:10 2022] PUT /identity/v3/projects/19a89aa5894d4b0f8034d04bdc4494d0/users/d085c6d872e94543b2bfd3e19311bf20/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:03:10.825100 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119825b4-1435-4033-8ab4-bd985e09c529 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:10.825391 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119825b4-1435-4033-8ab4-bd985e09c529 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:10.825670 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119825b4-1435-4033-8ab4-bd985e09c529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:10.826881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e4aea2-1c0d-4e3f-b1c0-24e742824cbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:10.829179 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e4aea2-1c0d-4e3f-b1c0-24e742824cbe service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:10.842767 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 944/1867] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:03:10.866652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-119825b4-1435-4033-8ab4-bd985e09c529 None None] MFA Rules not processed for user `d085c6d872e94543b2bfd3e19311bf20`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:10.867898 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-119825b4-1435-4033-8ab4-bd985e09c529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:11.012384 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 924/1868] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:10 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:11.030762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4aea2-1c0d-4e3f-b1c0-24e742824cbe service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:11.033164 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-74e4aea2-1c0d-4e3f-b1c0-24e742824cbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:11.064225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:11.120196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4aea2-1c0d-4e3f-b1c0-24e742824cbe service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:11.178420 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e64a0-6edf-4527-9b13-6c54fae65a2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:11.179066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e64a0-6edf-4527-9b13-6c54fae65a2b service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:11.179349 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e64a0-6edf-4527-9b13-6c54fae65a2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:11.179625 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e64a0-6edf-4527-9b13-6c54fae65a2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:11.192423 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1e64a0-6edf-4527-9b13-6c54fae65a2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:11.193939 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1e64a0-6edf-4527-9b13-6c54fae65a2b service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:11.209181 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 945/1869] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:11 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:03:11.424046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119825b4-1435-4033-8ab4-bd985e09c529 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:11.425593 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-119825b4-1435-4033-8ab4-bd985e09c529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:11.448466 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119825b4-1435-4033-8ab4-bd985e09c529 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:11.458298 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b40d0b-0d59-4419-a3de-49a3a29ee962 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:11.459314 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b40d0b-0d59-4419-a3de-49a3a29ee962 service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:11.460727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b40d0b-0d59-4419-a3de-49a3a29ee962 service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:11.460727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b40d0b-0d59-4419-a3de-49a3a29ee962 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:11.466784 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-77b40d0b-0d59-4419-a3de-49a3a29ee962 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:11.529500 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e64a0-6edf-4527-9b13-6c54fae65a2b service cinder] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:11.531808 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-cc1e64a0-6edf-4527-9b13-6c54fae65a2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:11.609701 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b40d0b-0d59-4419-a3de-49a3a29ee962 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:11.612818 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b40d0b-0d59-4419-a3de-49a3a29ee962 service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:11.646615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 925/1870] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:03:11.648894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b40d0b-0d59-4419-a3de-49a3a29ee962 service cinder] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:11.650835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-77b40d0b-0d59-4419-a3de-49a3a29ee962 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:11.741078 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:11.753735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e64a0-6edf-4527-9b13-6c54fae65a2b service cinder] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:11.766334 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4206fcb-2505-4228-adc3-3bcdb0f31368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:11.767187 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4206fcb-2505-4228-adc3-3bcdb0f31368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:11.767299 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4206fcb-2505-4228-adc3-3bcdb0f31368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:11.767713 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4206fcb-2505-4228-adc3-3bcdb0f31368 admin admin] PATH_INFO: `/v3/projects/31443f67c6b74defa673e3165414137e` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:11.777349 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4206fcb-2505-4228-adc3-3bcdb0f31368 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31443f67c6b74defa673e3165414137e)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:11.779269 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4206fcb-2505-4228-adc3-3bcdb0f31368 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:11.780497 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b40d0b-0d59-4419-a3de-49a3a29ee962 service cinder] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:11.788146 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9719014a-dd91-4e8a-8545-a42cd2c31ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:11.789157 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719014a-dd91-4e8a-8545-a42cd2c31ffb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:11.789943 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719014a-dd91-4e8a-8545-a42cd2c31ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:11.790499 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719014a-dd91-4e8a-8545-a42cd2c31ffb admin admin] PATH_INFO: `/v3/projects/63a69467039e43e29d1a684b31ee814f` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:11.792166 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9719014a-dd91-4e8a-8545-a42cd2c31ffb admin admin] RBAC: Authorizing `identity:delete_project(project_id=63a69467039e43e29d1a684b31ee814f)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:11.794960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9719014a-dd91-4e8a-8545-a42cd2c31ffb admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:11.856545 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-e4206fcb-2505-4228-adc3-3bcdb0f31368 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31443f67c6b74defa673e3165414137e', 'request_id': 'req-e4206fcb-2505-4228-adc3-3bcdb0f31368'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:11.867761 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-e4206fcb-2505-4228-adc3-3bcdb0f31368 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31443f67c6b74defa673e3165414137e', 'request_id': 'req-e4206fcb-2505-4228-adc3-3bcdb0f31368'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:11.882430 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-e4206fcb-2505-4228-adc3-3bcdb0f31368 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31443f67c6b74defa673e3165414137e', 'request_id': 'req-e4206fcb-2505-4228-adc3-3bcdb0f31368'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:11.885993 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-9719014a-dd91-4e8a-8545-a42cd2c31ffb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63a69467039e43e29d1a684b31ee814f', 'request_id': 'req-9719014a-dd91-4e8a-8545-a42cd2c31ffb'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:11.905839 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-9719014a-dd91-4e8a-8545-a42cd2c31ffb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63a69467039e43e29d1a684b31ee814f', 'request_id': 'req-9719014a-dd91-4e8a-8545-a42cd2c31ffb'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:11.915885 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-e4206fcb-2505-4228-adc3-3bcdb0f31368 admin admin] The token cache is being invalidate because project 31443f67c6b74defa673e3165414137e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:03:11.918221 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-e4206fcb-2505-4228-adc3-3bcdb0f31368 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:11.919120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-e4206fcb-2505-4228-adc3-3bcdb0f31368 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:11.919312 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-9719014a-dd91-4e8a-8545-a42cd2c31ffb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63a69467039e43e29d1a684b31ee814f', 'request_id': 'req-9719014a-dd91-4e8a-8545-a42cd2c31ffb'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:11.930392 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 946/1871] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:03:11 2022] DELETE /identity/v3/projects/31443f67c6b74defa673e3165414137e => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:11.933085 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072bdc0e-d472-4a33-bf75-5446485b02f8 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:11.933568 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072bdc0e-d472-4a33-bf75-5446485b02f8 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:11.934038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072bdc0e-d472-4a33-bf75-5446485b02f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:11.940955 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-9719014a-dd91-4e8a-8545-a42cd2c31ffb admin admin] The token cache is being invalidate because project 63a69467039e43e29d1a684b31ee814f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:03:11.941597 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-9719014a-dd91-4e8a-8545-a42cd2c31ffb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:11.942356 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-9719014a-dd91-4e8a-8545-a42cd2c31ffb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:11.958491 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 926/1872] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:03:11 2022] DELETE /identity/v3/projects/63a69467039e43e29d1a684b31ee814f => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:11.961505 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd10882-2d0a-40d4-bb49-65d37f0ae920 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:11.961807 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd10882-2d0a-40d4-bb49-65d37f0ae920 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:11.962246 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd10882-2d0a-40d4-bb49-65d37f0ae920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:11.964489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:11.973350 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-072bdc0e-d472-4a33-bf75-5446485b02f8 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:11.976020 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-072bdc0e-d472-4a33-bf75-5446485b02f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:11.994046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-4cd10882-2d0a-40d4-bb49-65d37f0ae920 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:11.995529 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-4cd10882-2d0a-40d4-bb49-65d37f0ae920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:12.100902 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 947/1873] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:12.101260 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 927/1874] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:12.102304 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02261e-9d4e-4ccf-8c55-55ef2b05fe78 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:12.102600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02261e-9d4e-4ccf-8c55-55ef2b05fe78 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:12.103002 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02261e-9d4e-4ccf-8c55-55ef2b05fe78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:12.114023 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d06d1e-390d-45a1-abef-f7453c4986a4 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:12.114315 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d06d1e-390d-45a1-abef-f7453c4986a4 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:12.114593 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d06d1e-390d-45a1-abef-f7453c4986a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:12.146824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-f1d06d1e-390d-45a1-abef-f7453c4986a4 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:12.148448 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-f1d06d1e-390d-45a1-abef-f7453c4986a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:12.164407 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 928/1875] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:12.165311 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d06d1e-390d-45a1-abef-f7453c4986a4 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:12.166762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-6e02261e-9d4e-4ccf-8c55-55ef2b05fe78 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:12.169821 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-6e02261e-9d4e-4ccf-8c55-55ef2b05fe78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:12.177303 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d06d1e-390d-45a1-abef-f7453c4986a4 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:12.183369 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c4eb3b-c9c0-49a5-849a-865d5242b2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:12.184128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 948/1876] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:03:12.184423 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4eb3b-c9c0-49a5-849a-865d5242b2da admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:12.184787 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4eb3b-c9c0-49a5-849a-865d5242b2da admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:12.185069 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4eb3b-c9c0-49a5-849a-865d5242b2da admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:12.185786 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c4eb3b-c9c0-49a5-849a-865d5242b2da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:03:12.186184 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c4eb3b-c9c0-49a5-849a-865d5242b2da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:12.187852 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c4eb3b-c9c0-49a5-849a-865d5242b2da admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:12.188691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e02261e-9d4e-4ccf-8c55-55ef2b05fe78 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:12.211554 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 929/1877] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:03:12.211554 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c4eb3b-c9c0-49a5-849a-865d5242b2da admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:12.211798 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e02261e-9d4e-4ccf-8c55-55ef2b05fe78 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:12.219467 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c4eb3b-c9c0-49a5-849a-865d5242b2da admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:12.220751 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21340e-d559-4ea8-a05a-588e1a6df952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:12.221392 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21340e-d559-4ea8-a05a-588e1a6df952 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:12.221731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21340e-d559-4ea8-a05a-588e1a6df952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:12.222032 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21340e-d559-4ea8-a05a-588e1a6df952 admin admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:12.222814 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b21340e-d559-4ea8-a05a-588e1a6df952 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:03:12.223205 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b21340e-d559-4ea8-a05a-588e1a6df952 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:12.225195 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b21340e-d559-4ea8-a05a-588e1a6df952 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:12.228530 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6dab3-d50a-4db7-b643-caf1cd42cedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:12.229829 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6dab3-d50a-4db7-b643-caf1cd42cedb admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:12.231077 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6dab3-d50a-4db7-b643-caf1cd42cedb admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:12.231687 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6dab3-d50a-4db7-b643-caf1cd42cedb admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:12.232769 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6dab3-d50a-4db7-b643-caf1cd42cedb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:12.235473 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6dab3-d50a-4db7-b643-caf1cd42cedb admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:12.241708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 949/1878] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:03:12.245929 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21340e-d559-4ea8-a05a-588e1a6df952 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:12.265023 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21340e-d559-4ea8-a05a-588e1a6df952 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:12.284079 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9b44b-fc39-4c94-8ade-6c8169b9db88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:12.284079 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9b44b-fc39-4c94-8ade-6c8169b9db88 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:12.284079 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9b44b-fc39-4c94-8ade-6c8169b9db88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:12.284079 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9b44b-fc39-4c94-8ade-6c8169b9db88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:12.285075 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9b44b-fc39-4c94-8ade-6c8169b9db88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:12.287889 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 930/1879] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:03:12 2022] POST /identity/v3/projects => generated 373 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:03:12.288792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9b44b-fc39-4c94-8ade-6c8169b9db88 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:12.292063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6dab3-d50a-4db7-b643-caf1cd42cedb admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:12.352881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 950/1880] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:03:12 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:12.357350 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9b44b-fc39-4c94-8ade-6c8169b9db88 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:12.383760 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:12.420191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6dab3-d50a-4db7-b643-caf1cd42cedb admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:12.455428 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9b44b-fc39-4c94-8ade-6c8169b9db88 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:12.467006 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d000fa-15c8-4e58-942f-717356a86fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:12.468614 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d000fa-15c8-4e58-942f-717356a86fd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:12.468997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d000fa-15c8-4e58-942f-717356a86fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:12.469295 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d000fa-15c8-4e58-942f-717356a86fd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:12.469984 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d000fa-15c8-4e58-942f-717356a86fd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:12.473639 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d000fa-15c8-4e58-942f-717356a86fd1 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:12.506158 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 951/1881] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:03:12 2022] POST /identity/v3/users => generated 288 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:03:12.507848 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d000fa-15c8-4e58-942f-717356a86fd1 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:12.519834 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99ebdcb-5cf0-456e-9607-0ff7575399a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:12.520757 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ebdcb-5cf0-456e-9607-0ff7575399a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:12.521229 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ebdcb-5cf0-456e-9607-0ff7575399a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:12.521470 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ebdcb-5cf0-456e-9607-0ff7575399a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:12.522089 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99ebdcb-5cf0-456e-9607-0ff7575399a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:12.522889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d000fa-15c8-4e58-942f-717356a86fd1 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:12.523999 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99ebdcb-5cf0-456e-9607-0ff7575399a5 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:12.530046 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68863cc8-50c1-4e83-b440-5eaea380e874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:12.530684 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68863cc8-50c1-4e83-b440-5eaea380e874 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:12.531002 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68863cc8-50c1-4e83-b440-5eaea380e874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:12.531390 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68863cc8-50c1-4e83-b440-5eaea380e874 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:12.532109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68863cc8-50c1-4e83-b440-5eaea380e874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:12.533814 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68863cc8-50c1-4e83-b440-5eaea380e874 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:12.550622 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 952/1882] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:12.552022 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68863cc8-50c1-4e83-b440-5eaea380e874 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:12.552866 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 931/1883] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:03:12 2022] POST /identity/v3/users => generated 287 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:03:12.553773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99ebdcb-5cf0-456e-9607-0ff7575399a5 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:12.568282 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68863cc8-50c1-4e83-b440-5eaea380e874 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:12.573731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f27380d-ba89-4104-8bea-140b0521ba2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:12.574310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27380d-ba89-4104-8bea-140b0521ba2a admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:12.574718 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27380d-ba89-4104-8bea-140b0521ba2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:12.574978 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27380d-ba89-4104-8bea-140b0521ba2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:12.576571 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99ebdcb-5cf0-456e-9607-0ff7575399a5 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:12.582052 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af97430-546b-45be-b6e8-32103b1663b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:12.582673 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af97430-546b-45be-b6e8-32103b1663b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:12.582950 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af97430-546b-45be-b6e8-32103b1663b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:12.583223 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af97430-546b-45be-b6e8-32103b1663b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:12.583922 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af97430-546b-45be-b6e8-32103b1663b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:12.585726 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af97430-546b-45be-b6e8-32103b1663b1 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:12.588994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f27380d-ba89-4104-8bea-140b0521ba2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:12.590724 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f27380d-ba89-4104-8bea-140b0521ba2a admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:12.607617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 932/1884] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:12.611352 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 953/1885] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:03:12.611784 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af97430-546b-45be-b6e8-32103b1663b1 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:12.612896 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f27380d-ba89-4104-8bea-140b0521ba2a admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:12.622890 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af97430-546b-45be-b6e8-32103b1663b1 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:12.625120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f27380d-ba89-4104-8bea-140b0521ba2a admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:12.629031 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf9b4d-4340-4447-9b22-ee2ff2652f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:12.629953 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf9b4d-4340-4447-9b22-ee2ff2652f59 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:12.630381 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf9b4d-4340-4447-9b22-ee2ff2652f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:12.630813 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf9b4d-4340-4447-9b22-ee2ff2652f59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:12.631654 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf9b4d-4340-4447-9b22-ee2ff2652f59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:12.631816 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed22706-0abd-4c53-9a46-f67f469d121d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:12.632530 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed22706-0abd-4c53-9a46-f67f469d121d admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:12.632908 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed22706-0abd-4c53-9a46-f67f469d121d admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:12.633185 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed22706-0abd-4c53-9a46-f67f469d121d admin admin] PATH_INFO: `/v3/projects/4ae21d4c475b4f56896aef81c29d4a30/users/c3caca5de8034d8dab6f226ade43267f/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:12.634904 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf9b4d-4340-4447-9b22-ee2ff2652f59 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:12.653108 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed22706-0abd-4c53-9a46-f67f469d121d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ae21d4c475b4f56896aef81c29d4a30, user_id=c3caca5de8034d8dab6f226ade43267f, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:12.656208 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed22706-0abd-4c53-9a46-f67f469d121d admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:12.672329 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 933/1887] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:12.677691 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf9b4d-4340-4447-9b22-ee2ff2652f59 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:12.680919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 954/1887] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:12 2022] PUT /identity/v3/projects/4ae21d4c475b4f56896aef81c29d4a30/users/c3caca5de8034d8dab6f226ade43267f/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:12.682609 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee33af-ede0-410f-b0bf-bfaa82c1a2e2 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:12.682987 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee33af-ede0-410f-b0bf-bfaa82c1a2e2 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:12.683325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee33af-ede0-410f-b0bf-bfaa82c1a2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:12.693486 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf9b4d-4340-4447-9b22-ee2ff2652f59 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:12.709730 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:12.722808 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-b3ee33af-ede0-410f-b0bf-bfaa82c1a2e2 None None] MFA Rules not processed for user `c3caca5de8034d8dab6f226ade43267f`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:12.725362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-b3ee33af-ede0-410f-b0bf-bfaa82c1a2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:12.793845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d0299-0401-4ac3-ada4-b64ee39b8f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:12.794784 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d0299-0401-4ac3-ada4-b64ee39b8f6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:12.797019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d0299-0401-4ac3-ada4-b64ee39b8f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:12.797466 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d0299-0401-4ac3-ada4-b64ee39b8f6e admin admin] PATH_INFO: `/v3/projects/37ce1647a63d44028a709829c6e91fed/users/fdca1b4d86984198a9c79c40098b5fa8/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:12.834113 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d0299-0401-4ac3-ada4-b64ee39b8f6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=37ce1647a63d44028a709829c6e91fed, user_id=fdca1b4d86984198a9c79c40098b5fa8, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:12.838064 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d0299-0401-4ac3-ada4-b64ee39b8f6e admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:12.860368 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 955/1888] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:12.865842 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee33af-ede0-410f-b0bf-bfaa82c1a2e2 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:12.866580 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 934/1889] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:12 2022] PUT /identity/v3/projects/37ce1647a63d44028a709829c6e91fed/users/fdca1b4d86984198a9c79c40098b5fa8/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:03:12.891968 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007c6af-23c1-4427-93e2-45ce10c08599 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:12.892300 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007c6af-23c1-4427-93e2-45ce10c08599 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:12.892539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007c6af-23c1-4427-93e2-45ce10c08599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:12.900896 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee33af-ede0-410f-b0bf-bfaa82c1a2e2 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:12.974922 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-e007c6af-23c1-4427-93e2-45ce10c08599 None None] MFA Rules not processed for user `fdca1b4d86984198a9c79c40098b5fa8`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:12.978368 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-e007c6af-23c1-4427-93e2-45ce10c08599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:13.028925 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:13.075999 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02db37-58fb-40e3-a29f-7a721a837e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:13.076870 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02db37-58fb-40e3-a29f-7a721a837e47 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:13.077384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02db37-58fb-40e3-a29f-7a721a837e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:13.077729 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02db37-58fb-40e3-a29f-7a721a837e47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:13.078551 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02db37-58fb-40e3-a29f-7a721a837e47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:13.081525 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02db37-58fb-40e3-a29f-7a721a837e47 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:13.107151 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 935/1890] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:13.108120 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e007c6af-23c1-4427-93e2-45ce10c08599 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:13.109658 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-e007c6af-23c1-4427-93e2-45ce10c08599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:13.129292 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e007c6af-23c1-4427-93e2-45ce10c08599 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:13.135127 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cbd3b8-769d-414a-a0ed-e172259bd850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:13.135893 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cbd3b8-769d-414a-a0ed-e172259bd850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:13.136171 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cbd3b8-769d-414a-a0ed-e172259bd850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:13.136848 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cbd3b8-769d-414a-a0ed-e172259bd850 admin admin] PATH_INFO: `/v3/users/ffb5b3a035b2463cb1f5bc055c82b09b` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:13.138429 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cbd3b8-769d-414a-a0ed-e172259bd850 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffb5b3a035b2463cb1f5bc055c82b09b)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:13.141432 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cbd3b8-769d-414a-a0ed-e172259bd850 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:13.164766 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 956/1891] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:03:12 2022] POST /identity/v3/projects => generated 373 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:03:13.167168 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02db37-58fb-40e3-a29f-7a721a837e47 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:13.211195 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-e7cbd3b8-769d-414a-a0ed-e172259bd850 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffb5b3a035b2463cb1f5bc055c82b09b'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:13.221246 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-e7cbd3b8-769d-414a-a0ed-e172259bd850 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffb5b3a035b2463cb1f5bc055c82b09b'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:13.236398 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-e7cbd3b8-769d-414a-a0ed-e172259bd850 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffb5b3a035b2463cb1f5bc055c82b09b'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:13.238674 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-e7cbd3b8-769d-414a-a0ed-e172259bd850 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffb5b3a035b2463cb1f5bc055c82b09b'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:13.246592 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-e7cbd3b8-769d-414a-a0ed-e172259bd850 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffb5b3a035b2463cb1f5bc055c82b09b'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:13.252846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02db37-58fb-40e3-a29f-7a721a837e47 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:13.255931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 936/1892] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:03:13 2022] DELETE /identity/v3/users/ffb5b3a035b2463cb1f5bc055c82b09b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:13.256878 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cbd3b8-769d-414a-a0ed-e172259bd850 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:13.260328 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa61640-aa16-4f9a-bc23-cf0051d44fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:13.260622 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-e7cbd3b8-769d-414a-a0ed-e172259bd850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:13.261408 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa61640-aa16-4f9a-bc23-cf0051d44fc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:13.261691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa61640-aa16-4f9a-bc23-cf0051d44fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:13.262135 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa61640-aa16-4f9a-bc23-cf0051d44fc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:13.262925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa61640-aa16-4f9a-bc23-cf0051d44fc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:13.265028 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa61640-aa16-4f9a-bc23-cf0051d44fc9 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:13.316109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 957/1893] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:03:13 2022] POST /identity/v3/projects => generated 369 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:13.316109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a5c5d-c376-4e30-b8f3-a01bc9fdee98 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:13.316109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a5c5d-c376-4e30-b8f3-a01bc9fdee98 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:13.316109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a5c5d-c376-4e30-b8f3-a01bc9fdee98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:13.341510 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-9a4a5c5d-c376-4e30-b8f3-a01bc9fdee98 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:13.342722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-9a4a5c5d-c376-4e30-b8f3-a01bc9fdee98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:13.350378 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:13.388127 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cbd3b8-769d-414a-a0ed-e172259bd850 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:13.431741 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 958/1894] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:13.433633 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a5c5d-c376-4e30-b8f3-a01bc9fdee98 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:13.434866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-9a4a5c5d-c376-4e30-b8f3-a01bc9fdee98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:13.461402 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a5c5d-c376-4e30-b8f3-a01bc9fdee98 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:13.467189 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe7722b-c4d2-425d-be02-04c46765bbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:13.467834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe7722b-c4d2-425d-be02-04c46765bbb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:13.468690 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:13.468825 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe7722b-c4d2-425d-be02-04c46765bbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:13.469377 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe7722b-c4d2-425d-be02-04c46765bbb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:13.471196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe7722b-c4d2-425d-be02-04c46765bbb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:13.474344 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe7722b-c4d2-425d-be02-04c46765bbb8 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:13.494663 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bfd09e-2b82-40ae-a1a6-8185cf245a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:13.495258 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bfd09e-2b82-40ae-a1a6-8185cf245a3a admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:13.495537 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bfd09e-2b82-40ae-a1a6-8185cf245a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:13.495858 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bfd09e-2b82-40ae-a1a6-8185cf245a3a admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:13.496607 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bfd09e-2b82-40ae-a1a6-8185cf245a3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:13.499230 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bfd09e-2b82-40ae-a1a6-8185cf245a3a admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:13.510197 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 959/1895] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:03:13 2022] POST /identity/v3/users => generated 291 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:13.512403 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe7722b-c4d2-425d-be02-04c46765bbb8 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:13.515475 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-ffe7722b-c4d2-425d-be02-04c46765bbb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:13.532612 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 937/1896] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:03:13 2022] POST /identity/v3/users => generated 293 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:13.533596 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bfd09e-2b82-40ae-a1a6-8185cf245a3a admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:13.544483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bfd09e-2b82-40ae-a1a6-8185cf245a3a admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:13.550610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b059a-7533-49b3-9b28-cab2ff6bc0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:13.551549 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b059a-7533-49b3-9b28-cab2ff6bc0fe admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:13.551827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b059a-7533-49b3-9b28-cab2ff6bc0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:13.552098 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b059a-7533-49b3-9b28-cab2ff6bc0fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:13.553053 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b059a-7533-49b3-9b28-cab2ff6bc0fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:13.554647 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b059a-7533-49b3-9b28-cab2ff6bc0fe admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:13.571650 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 938/1897] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:13.574152 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b059a-7533-49b3-9b28-cab2ff6bc0fe admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:13.587214 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b059a-7533-49b3-9b28-cab2ff6bc0fe admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:13.592148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c24fa-0040-4254-81e4-22324248dc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:13.593304 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c24fa-0040-4254-81e4-22324248dc23 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:13.593304 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c24fa-0040-4254-81e4-22324248dc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:13.593617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c24fa-0040-4254-81e4-22324248dc23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:13.594317 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638c24fa-0040-4254-81e4-22324248dc23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:13.596055 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638c24fa-0040-4254-81e4-22324248dc23 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:13.611548 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 939/1898] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:13.611548 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c24fa-0040-4254-81e4-22324248dc23 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:13.620286 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c24fa-0040-4254-81e4-22324248dc23 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:13.623666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe7722b-c4d2-425d-be02-04c46765bbb8 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:13.626023 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741713d2-5b4d-4f7f-b0f7-596b5f742b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:13.626772 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741713d2-5b4d-4f7f-b0f7-596b5f742b81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:13.627216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741713d2-5b4d-4f7f-b0f7-596b5f742b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:13.627562 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741713d2-5b4d-4f7f-b0f7-596b5f742b81 admin admin] PATH_INFO: `/v3/projects/3e6d064e6ebd4cef94edc8afed451f38/users/cdcf1d15ca31458e9ce1b9c4d30309d1/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:13.628130 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3086b39-ac36-4a3c-8854-93788a81bb05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:13.629053 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3086b39-ac36-4a3c-8854-93788a81bb05 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:13.629551 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3086b39-ac36-4a3c-8854-93788a81bb05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:13.630016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3086b39-ac36-4a3c-8854-93788a81bb05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:13.641076 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3086b39-ac36-4a3c-8854-93788a81bb05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:13.645757 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3086b39-ac36-4a3c-8854-93788a81bb05 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:13.647778 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741713d2-5b4d-4f7f-b0f7-596b5f742b81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e6d064e6ebd4cef94edc8afed451f38, user_id=cdcf1d15ca31458e9ce1b9c4d30309d1, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:13.650726 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741713d2-5b4d-4f7f-b0f7-596b5f742b81 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:13.664702 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 960/1899] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:03:13.666165 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3086b39-ac36-4a3c-8854-93788a81bb05 service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:13.666293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 940/1900] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:13 2022] PUT /identity/v3/projects/3e6d064e6ebd4cef94edc8afed451f38/users/cdcf1d15ca31458e9ce1b9c4d30309d1/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:13.669858 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741713d2-5b4d-4f7f-b0f7-596b5f742b81 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:13.683324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3086b39-ac36-4a3c-8854-93788a81bb05 service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:13.772160 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741713d2-5b4d-4f7f-b0f7-596b5f742b81 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:13.777465 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71671a59-7eb0-4079-a172-df7962340a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:13.778088 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71671a59-7eb0-4079-a172-df7962340a95 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:13.778450 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71671a59-7eb0-4079-a172-df7962340a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:13.778718 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71671a59-7eb0-4079-a172-df7962340a95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:13.779460 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71671a59-7eb0-4079-a172-df7962340a95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:13.781402 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71671a59-7eb0-4079-a172-df7962340a95 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:13.796511 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 941/1901] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:03:13.797307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66615dd-a675-402d-a112-2b0028cda530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:13.798060 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71671a59-7eb0-4079-a172-df7962340a95 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:13.798185 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66615dd-a675-402d-a112-2b0028cda530 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:13.798428 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66615dd-a675-402d-a112-2b0028cda530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:13.798768 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66615dd-a675-402d-a112-2b0028cda530 admin admin] PATH_INFO: `/v3/projects/30817664bd81428ca8cbbe436257838e/users/faeb7733ef744a3392c2339c570515eb/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:13.811175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71671a59-7eb0-4079-a172-df7962340a95 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:13.816550 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a09b8-3505-4d77-84ed-7e44d3da4adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:13.817257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a09b8-3505-4d77-84ed-7e44d3da4adf admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:13.817545 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a09b8-3505-4d77-84ed-7e44d3da4adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:13.817822 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a09b8-3505-4d77-84ed-7e44d3da4adf admin admin] PATH_INFO: `/v3/projects/3e6d064e6ebd4cef94edc8afed451f38/users/cdcf1d15ca31458e9ce1b9c4d30309d1/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:13.819788 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a09b8-3505-4d77-84ed-7e44d3da4adf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e6d064e6ebd4cef94edc8afed451f38, user_id=cdcf1d15ca31458e9ce1b9c4d30309d1, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:13.819956 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66615dd-a675-402d-a112-2b0028cda530 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30817664bd81428ca8cbbe436257838e, user_id=faeb7733ef744a3392c2339c570515eb, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:13.822481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a09b8-3505-4d77-84ed-7e44d3da4adf admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:13.822791 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66615dd-a675-402d-a112-2b0028cda530 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:13.837599 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 942/1902] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:13 2022] PUT /identity/v3/projects/3e6d064e6ebd4cef94edc8afed451f38/users/cdcf1d15ca31458e9ce1b9c4d30309d1/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:13.851304 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 961/1903] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:13 2022] PUT /identity/v3/projects/30817664bd81428ca8cbbe436257838e/users/faeb7733ef744a3392c2339c570515eb/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:13.856758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a09b8-3505-4d77-84ed-7e44d3da4adf admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:13.857668 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b168a-c065-4da6-b2f8-abbd8b22dbe7 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:13.857951 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b168a-c065-4da6-b2f8-abbd8b22dbe7 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:13.858222 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b168a-c065-4da6-b2f8-abbd8b22dbe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:13.893241 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-1a4b168a-c065-4da6-b2f8-abbd8b22dbe7 None None] MFA Rules not processed for user `cdcf1d15ca31458e9ce1b9c4d30309d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:13.896922 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-1a4b168a-c065-4da6-b2f8-abbd8b22dbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:13.904925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:13.928143 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a09b8-3505-4d77-84ed-7e44d3da4adf admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:13.945971 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:13.981827 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 962/1904] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:13 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:13.997660 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b168a-c065-4da6-b2f8-abbd8b22dbe7 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:13.999219 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-1a4b168a-c065-4da6-b2f8-abbd8b22dbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:14.014351 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba465883-b5da-4f94-bf8f-eb3d0db8e9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:14.015134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba465883-b5da-4f94-bf8f-eb3d0db8e9a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:14.015474 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba465883-b5da-4f94-bf8f-eb3d0db8e9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:14.016262 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba465883-b5da-4f94-bf8f-eb3d0db8e9a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:14.017076 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba465883-b5da-4f94-bf8f-eb3d0db8e9a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:14.019355 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba465883-b5da-4f94-bf8f-eb3d0db8e9a4 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:14.032553 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 943/1905] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:03:14.033786 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba465883-b5da-4f94-bf8f-eb3d0db8e9a4 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:14.037234 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-ba465883-b5da-4f94-bf8f-eb3d0db8e9a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:14.051050 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba465883-b5da-4f94-bf8f-eb3d0db8e9a4 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:14.056018 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4bd7c7-a946-49e0-ad2a-45379429d761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:14.056708 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4bd7c7-a946-49e0-ad2a-45379429d761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:14.057020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4bd7c7-a946-49e0-ad2a-45379429d761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:14.057293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4bd7c7-a946-49e0-ad2a-45379429d761 admin admin] PATH_INFO: `/v3/projects/0525078eee2246d8826bd746fb4c2d72` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:14.058428 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4bd7c7-a946-49e0-ad2a-45379429d761 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0525078eee2246d8826bd746fb4c2d72)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:14.060422 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4bd7c7-a946-49e0-ad2a-45379429d761 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:14.062344 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b168a-c065-4da6-b2f8-abbd8b22dbe7 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:14.066199 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f385e4-51fe-4d76-b26e-1ba3e3105a3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:14.066792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f385e4-51fe-4d76-b26e-1ba3e3105a3a service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:14.067367 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f385e4-51fe-4d76-b26e-1ba3e3105a3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:14.067367 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f385e4-51fe-4d76-b26e-1ba3e3105a3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:14.076842 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f385e4-51fe-4d76-b26e-1ba3e3105a3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:14.078390 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f385e4-51fe-4d76-b26e-1ba3e3105a3a service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:14.088738 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 963/1906] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:13 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:03:14.089523 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f385e4-51fe-4d76-b26e-1ba3e3105a3a service cinder] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:14.100890 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f385e4-51fe-4d76-b26e-1ba3e3105a3a service cinder] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:14.120819 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-6d4bd7c7-a946-49e0-ad2a-45379429d761 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0525078eee2246d8826bd746fb4c2d72', 'request_id': 'req-6d4bd7c7-a946-49e0-ad2a-45379429d761'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:14.136838 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-6d4bd7c7-a946-49e0-ad2a-45379429d761 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0525078eee2246d8826bd746fb4c2d72', 'request_id': 'req-6d4bd7c7-a946-49e0-ad2a-45379429d761'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:14.144598 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-6d4bd7c7-a946-49e0-ad2a-45379429d761 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0525078eee2246d8826bd746fb4c2d72', 'request_id': 'req-6d4bd7c7-a946-49e0-ad2a-45379429d761'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:14.162712 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-6d4bd7c7-a946-49e0-ad2a-45379429d761 admin admin] The token cache is being invalidate because project 0525078eee2246d8826bd746fb4c2d72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:03:14.163015 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-6d4bd7c7-a946-49e0-ad2a-45379429d761 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:14.163843 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-6d4bd7c7-a946-49e0-ad2a-45379429d761 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:14.168616 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 944/1907] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:03:14 2022] DELETE /identity/v3/projects/0525078eee2246d8826bd746fb4c2d72 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:14.169852 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4bd7c7-a946-49e0-ad2a-45379429d761 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:14.171684 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-6d4bd7c7-a946-49e0-ad2a-45379429d761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:14.177504 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f8f95-6f30-4f53-8b35-7ac3bf1f26c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:14.178475 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f8f95-6f30-4f53-8b35-7ac3bf1f26c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:14.178799 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f8f95-6f30-4f53-8b35-7ac3bf1f26c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:14.179077 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f8f95-6f30-4f53-8b35-7ac3bf1f26c4 admin admin] PATH_INFO: `/v3/projects/30817664bd81428ca8cbbe436257838e/users/faeb7733ef744a3392c2339c570515eb/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:14.181142 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f8f95-6f30-4f53-8b35-7ac3bf1f26c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30817664bd81428ca8cbbe436257838e, user_id=faeb7733ef744a3392c2339c570515eb, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:14.183833 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f8f95-6f30-4f53-8b35-7ac3bf1f26c4 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:14.185806 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4bd7c7-a946-49e0-ad2a-45379429d761 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:14.192314 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b6d39-65cf-46d8-aa83-639b2b774300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:14.193072 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b6d39-65cf-46d8-aa83-639b2b774300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:14.193355 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b6d39-65cf-46d8-aa83-639b2b774300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:14.193632 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b6d39-65cf-46d8-aa83-639b2b774300 admin admin] PATH_INFO: `/v3/users/c8c4abac652f4542a08ea6e54c6a9206` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:14.194816 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 964/1908] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:14 2022] PUT /identity/v3/projects/30817664bd81428ca8cbbe436257838e/users/faeb7733ef744a3392c2339c570515eb/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:14.195013 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726b6d39-65cf-46d8-aa83-639b2b774300 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8c4abac652f4542a08ea6e54c6a9206)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:14.195689 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f8f95-6f30-4f53-8b35-7ac3bf1f26c4 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:14.196760 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726b6d39-65cf-46d8-aa83-639b2b774300 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:14.205928 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-b09f8f95-6f30-4f53-8b35-7ac3bf1f26c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:14.266475 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:14.272144 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-726b6d39-65cf-46d8-aa83-639b2b774300 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8c4abac652f4542a08ea6e54c6a9206'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:14.281500 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-726b6d39-65cf-46d8-aa83-639b2b774300 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8c4abac652f4542a08ea6e54c6a9206'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:14.286854 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f8f95-6f30-4f53-8b35-7ac3bf1f26c4 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:14.291552 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c06561-7841-494e-84b5-a539dd340449 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:14.292453 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c06561-7841-494e-84b5-a539dd340449 service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:14.292821 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c06561-7841-494e-84b5-a539dd340449 service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:14.293111 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c06561-7841-494e-84b5-a539dd340449 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:14.295422 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-c6c06561-7841-494e-84b5-a539dd340449 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:14.296612 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-726b6d39-65cf-46d8-aa83-639b2b774300 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8c4abac652f4542a08ea6e54c6a9206'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:14.297303 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-726b6d39-65cf-46d8-aa83-639b2b774300 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8c4abac652f4542a08ea6e54c6a9206'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:14.305765 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-726b6d39-65cf-46d8-aa83-639b2b774300 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8c4abac652f4542a08ea6e54c6a9206'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:14.313086 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 945/1909] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:03:14 2022] DELETE /identity/v3/users/c8c4abac652f4542a08ea6e54c6a9206 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:03:14.318028 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401fbc8-b030-483a-8fff-36c33911254a None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:14.318288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401fbc8-b030-483a-8fff-36c33911254a None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:14.318571 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401fbc8-b030-483a-8fff-36c33911254a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:14.332151 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:14.359479 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-e401fbc8-b030-483a-8fff-36c33911254a None None] MFA Rules not processed for user `faeb7733ef744a3392c2339c570515eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:14.364714 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-e401fbc8-b030-483a-8fff-36c33911254a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:14.386808 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c06561-7841-494e-84b5-a539dd340449 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:14.390139 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c06561-7841-494e-84b5-a539dd340449 service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:14.413999 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 965/1910] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:03:14.481427 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 946/1911] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:14.526763 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c06561-7841-494e-84b5-a539dd340449 service cinder] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:14.528451 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-c6c06561-7841-494e-84b5-a539dd340449 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:14.546861 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c06561-7841-494e-84b5-a539dd340449 service cinder] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:14.609493 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302fa9a6-2a08-4189-9f3b-ee98f48f2c34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:14.610389 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302fa9a6-2a08-4189-9f3b-ee98f48f2c34 service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:14.610902 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302fa9a6-2a08-4189-9f3b-ee98f48f2c34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:14.611343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302fa9a6-2a08-4189-9f3b-ee98f48f2c34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:14.614947 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-302fa9a6-2a08-4189-9f3b-ee98f48f2c34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:14.703593 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302fa9a6-2a08-4189-9f3b-ee98f48f2c34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:14.707034 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302fa9a6-2a08-4189-9f3b-ee98f48f2c34 service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:14.720770 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 966/1912] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:03:14.858109 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401fbc8-b030-483a-8fff-36c33911254a None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:14.860143 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-e401fbc8-b030-483a-8fff-36c33911254a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:14.961562 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401fbc8-b030-483a-8fff-36c33911254a None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:14.968992 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bce55a-e337-4312-9a88-8d225848fdeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:14.969638 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bce55a-e337-4312-9a88-8d225848fdeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:14.969928 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bce55a-e337-4312-9a88-8d225848fdeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:14.970491 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bce55a-e337-4312-9a88-8d225848fdeb admin admin] PATH_INFO: `/v3/projects/4d879df100d4411ba8e9e750380e6857` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:14.972680 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bce55a-e337-4312-9a88-8d225848fdeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d879df100d4411ba8e9e750380e6857)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:14.974229 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bce55a-e337-4312-9a88-8d225848fdeb admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:15.045125 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-94bce55a-e337-4312-9a88-8d225848fdeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d879df100d4411ba8e9e750380e6857', 'request_id': 'req-94bce55a-e337-4312-9a88-8d225848fdeb'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:15.053879 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-94bce55a-e337-4312-9a88-8d225848fdeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d879df100d4411ba8e9e750380e6857', 'request_id': 'req-94bce55a-e337-4312-9a88-8d225848fdeb'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:15.061633 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-94bce55a-e337-4312-9a88-8d225848fdeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d879df100d4411ba8e9e750380e6857', 'request_id': 'req-94bce55a-e337-4312-9a88-8d225848fdeb'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:15.074188 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-94bce55a-e337-4312-9a88-8d225848fdeb admin admin] The token cache is being invalidate because project 4d879df100d4411ba8e9e750380e6857 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:03:15.074408 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-94bce55a-e337-4312-9a88-8d225848fdeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:15.075037 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-94bce55a-e337-4312-9a88-8d225848fdeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:15.077562 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 947/1913] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:03:14 2022] DELETE /identity/v3/projects/4d879df100d4411ba8e9e750380e6857 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:15.082823 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0c937-3775-46de-829f-de5143084d5b None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:15.083314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0c937-3775-46de-829f-de5143084d5b None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:15.083473 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0c937-3775-46de-829f-de5143084d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:15.110996 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-04f0c937-3775-46de-829f-de5143084d5b None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:15.112837 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-04f0c937-3775-46de-829f-de5143084d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:15.173428 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:15.193220 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 967/1914] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:03:15.200412 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc17361-4f44-4c55-a91a-a43a69f87e45 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:15.200664 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc17361-4f44-4c55-a91a-a43a69f87e45 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:15.201058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc17361-4f44-4c55-a91a-a43a69f87e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:15.228312 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-cfc17361-4f44-4c55-a91a-a43a69f87e45 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:15.229738 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-cfc17361-4f44-4c55-a91a-a43a69f87e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:15.239795 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 948/1915] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:15.244091 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0c937-3775-46de-829f-de5143084d5b None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:15.257222 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0c937-3775-46de-829f-de5143084d5b None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:15.261939 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded97aa-9aa0-4e4c-9cd5-98dc4eca5348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:15.262554 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded97aa-9aa0-4e4c-9cd5-98dc4eca5348 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:15.262839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded97aa-9aa0-4e4c-9cd5-98dc4eca5348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:15.263118 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded97aa-9aa0-4e4c-9cd5-98dc4eca5348 admin admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:15.263839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cded97aa-9aa0-4e4c-9cd5-98dc4eca5348 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:03:15.264149 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cded97aa-9aa0-4e4c-9cd5-98dc4eca5348 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:15.265850 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cded97aa-9aa0-4e4c-9cd5-98dc4eca5348 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:15.280665 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 968/1916] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:03:15.285048 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc17361-4f44-4c55-a91a-a43a69f87e45 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:15.298400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc17361-4f44-4c55-a91a-a43a69f87e45 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:15.304615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02291da8-83ce-44e9-b471-4e61c162a2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:15.305358 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02291da8-83ce-44e9-b471-4e61c162a2e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:15.305682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02291da8-83ce-44e9-b471-4e61c162a2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:15.305958 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02291da8-83ce-44e9-b471-4e61c162a2e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:15.306628 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02291da8-83ce-44e9-b471-4e61c162a2e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:15.308539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02291da8-83ce-44e9-b471-4e61c162a2e8 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:15.325918 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:15.342211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 949/1917] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:03:15 2022] POST /identity/v3/projects => generated 389 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:15.345595 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded97aa-9aa0-4e4c-9cd5-98dc4eca5348 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:15.355654 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded97aa-9aa0-4e4c-9cd5-98dc4eca5348 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:15.416687 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20bc991-aede-4b9b-9ddd-da662b76e0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:15.417311 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20bc991-aede-4b9b-9ddd-da662b76e0dd admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:15.417603 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20bc991-aede-4b9b-9ddd-da662b76e0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:15.417873 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20bc991-aede-4b9b-9ddd-da662b76e0dd admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:15.418540 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20bc991-aede-4b9b-9ddd-da662b76e0dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:15.420881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20bc991-aede-4b9b-9ddd-da662b76e0dd admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:15.445642 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 969/1918] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:03:15 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:15.449741 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02291da8-83ce-44e9-b471-4e61c162a2e8 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:15.460626 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02291da8-83ce-44e9-b471-4e61c162a2e8 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:15.465154 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b34b0-a333-43cf-b78a-ed798622fec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:15.465768 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b34b0-a333-43cf-b78a-ed798622fec9 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:15.466054 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b34b0-a333-43cf-b78a-ed798622fec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:15.466330 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b34b0-a333-43cf-b78a-ed798622fec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:15.467044 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b34b0-a333-43cf-b78a-ed798622fec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:15.468832 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b34b0-a333-43cf-b78a-ed798622fec9 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:15.480360 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 950/1919] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:15.484533 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20bc991-aede-4b9b-9ddd-da662b76e0dd admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:15.495535 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20bc991-aede-4b9b-9ddd-da662b76e0dd admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:15.500202 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a714e41-12db-4869-b3df-75168ff3481d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:15.500855 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a714e41-12db-4869-b3df-75168ff3481d admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:15.501140 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a714e41-12db-4869-b3df-75168ff3481d admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:15.501415 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a714e41-12db-4869-b3df-75168ff3481d admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:15.502110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a714e41-12db-4869-b3df-75168ff3481d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:15.503921 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a714e41-12db-4869-b3df-75168ff3481d admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:15.517212 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 970/1920] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:15.518702 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b34b0-a333-43cf-b78a-ed798622fec9 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:15.530347 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b34b0-a333-43cf-b78a-ed798622fec9 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:15.536342 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe93ca-dd85-413e-bc41-9e9eabb79a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:15.537119 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe93ca-dd85-413e-bc41-9e9eabb79a1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:15.537401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe93ca-dd85-413e-bc41-9e9eabb79a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:15.537682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe93ca-dd85-413e-bc41-9e9eabb79a1d admin admin] PATH_INFO: `/v3/projects/6e24b112a9bb4e529b1fe3c8be573ed4/users/9799d7c72ad540eb9dafacdeefb595a3/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:15.559495 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfe93ca-dd85-413e-bc41-9e9eabb79a1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e24b112a9bb4e529b1fe3c8be573ed4, user_id=9799d7c72ad540eb9dafacdeefb595a3, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:15.562487 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfe93ca-dd85-413e-bc41-9e9eabb79a1d admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:15.579618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 951/1921] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:15 2022] PUT /identity/v3/projects/6e24b112a9bb4e529b1fe3c8be573ed4/users/9799d7c72ad540eb9dafacdeefb595a3/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:15.590230 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf9163-c3fd-4c7c-bcfc-1bbd2dd837b8 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:15.590509 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf9163-c3fd-4c7c-bcfc-1bbd2dd837b8 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:15.590782 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf9163-c3fd-4c7c-bcfc-1bbd2dd837b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:15.624564 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-5acf9163-c3fd-4c7c-bcfc-1bbd2dd837b8 None None] MFA Rules not processed for user `9799d7c72ad540eb9dafacdeefb595a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:15.625904 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5acf9163-c3fd-4c7c-bcfc-1bbd2dd837b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:15.727436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 971/1922] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:15 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:15.732096 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe93ca-dd85-413e-bc41-9e9eabb79a1d admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:15.797555 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf9163-c3fd-4c7c-bcfc-1bbd2dd837b8 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:15.799697 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5acf9163-c3fd-4c7c-bcfc-1bbd2dd837b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:15.840217 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe93ca-dd85-413e-bc41-9e9eabb79a1d admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:15.851347 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f954358-b593-4845-81cc-d1ea3821d6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:15.852188 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f954358-b593-4845-81cc-d1ea3821d6b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:15.852537 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f954358-b593-4845-81cc-d1ea3821d6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:15.853054 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f954358-b593-4845-81cc-d1ea3821d6b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:15.853827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f954358-b593-4845-81cc-d1ea3821d6b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:15.856838 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f954358-b593-4845-81cc-d1ea3821d6b7 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:15.901041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:15.917373 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 952/1923] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:03:15 2022] POST /identity/v3/projects => generated 391 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:15.921921 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f954358-b593-4845-81cc-d1ea3821d6b7 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:15.953772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf9163-c3fd-4c7c-bcfc-1bbd2dd837b8 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:16.040007 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f954358-b593-4845-81cc-d1ea3821d6b7 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:16.049912 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea3f13-3b03-4e0a-9a75-564af8b5da21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:16.051107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea3f13-3b03-4e0a-9a75-564af8b5da21 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:16.051504 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea3f13-3b03-4e0a-9a75-564af8b5da21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:16.051852 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea3f13-3b03-4e0a-9a75-564af8b5da21 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:16.052619 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea3f13-3b03-4e0a-9a75-564af8b5da21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:16.054735 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea3f13-3b03-4e0a-9a75-564af8b5da21 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:16.070378 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac0c44-b22b-45c6-bca8-7f3b69478ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:16.071035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac0c44-b22b-45c6-bca8-7f3b69478ec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:16.071350 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac0c44-b22b-45c6-bca8-7f3b69478ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:16.073161 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac0c44-b22b-45c6-bca8-7f3b69478ec0 admin admin] PATH_INFO: `/v3/users/886a28c0adae435998242d5a4053e2c4` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:16.074628 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cac0c44-b22b-45c6-bca8-7f3b69478ec0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=886a28c0adae435998242d5a4053e2c4)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:16.077867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cac0c44-b22b-45c6-bca8-7f3b69478ec0 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:16.090206 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 953/1924] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:03:15 2022] POST /identity/v3/users => generated 302 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:16.095699 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea3f13-3b03-4e0a-9a75-564af8b5da21 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:16.118801 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea3f13-3b03-4e0a-9a75-564af8b5da21 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:16.132941 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73a5c02-e59e-441c-9044-353bde63e219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:16.132941 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a5c02-e59e-441c-9044-353bde63e219 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:16.132941 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a5c02-e59e-441c-9044-353bde63e219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:16.132941 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a5c02-e59e-441c-9044-353bde63e219 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:16.133242 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73a5c02-e59e-441c-9044-353bde63e219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:16.138446 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73a5c02-e59e-441c-9044-353bde63e219 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:16.155896 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 954/1925] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:03:16.160931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73a5c02-e59e-441c-9044-353bde63e219 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:16.164066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5cac0c44-b22b-45c6-bca8-7f3b69478ec0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '886a28c0adae435998242d5a4053e2c4'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:16.174341 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73a5c02-e59e-441c-9044-353bde63e219 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:16.181612 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edf812b-d42f-4df6-8969-9fb6310f2c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:16.182383 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf812b-d42f-4df6-8969-9fb6310f2c67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:16.182735 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf812b-d42f-4df6-8969-9fb6310f2c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:16.183139 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf812b-d42f-4df6-8969-9fb6310f2c67 admin admin] PATH_INFO: `/v3/projects/3c989e137c3b4803929fdb0433419c60/users/b3d2f631881e46b5bffaae677c1dad90/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:16.189842 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5cac0c44-b22b-45c6-bca8-7f3b69478ec0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '886a28c0adae435998242d5a4053e2c4'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:16.207626 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edf812b-d42f-4df6-8969-9fb6310f2c67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c989e137c3b4803929fdb0433419c60, user_id=b3d2f631881e46b5bffaae677c1dad90, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:16.207626 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edf812b-d42f-4df6-8969-9fb6310f2c67 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:16.211397 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5cac0c44-b22b-45c6-bca8-7f3b69478ec0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '886a28c0adae435998242d5a4053e2c4'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:16.216628 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:16.226261 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 955/1926] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:16 2022] PUT /identity/v3/projects/3c989e137c3b4803929fdb0433419c60/users/b3d2f631881e46b5bffaae677c1dad90/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:03:16.229585 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5cac0c44-b22b-45c6-bca8-7f3b69478ec0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '886a28c0adae435998242d5a4053e2c4'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:16.231052 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5cac0c44-b22b-45c6-bca8-7f3b69478ec0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '886a28c0adae435998242d5a4053e2c4'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:16.231772 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edf812b-d42f-4df6-8969-9fb6310f2c67 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:16.234444 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-3edf812b-d42f-4df6-8969-9fb6310f2c67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:16.242653 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 972/1927] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:03:15 2022] DELETE /identity/v3/users/886a28c0adae435998242d5a4053e2c4 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:16.244271 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbdd1f-a2f0-4003-b21a-8f300e7d9cb0 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:16.245158 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbdd1f-a2f0-4003-b21a-8f300e7d9cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:16.245158 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbdd1f-a2f0-4003-b21a-8f300e7d9cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:16.269392 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-6fcbdd1f-a2f0-4003-b21a-8f300e7d9cb0 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:16.271023 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-6fcbdd1f-a2f0-4003-b21a-8f300e7d9cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:16.341412 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edf812b-d42f-4df6-8969-9fb6310f2c67 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:16.389329 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 973/1928] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:16.408599 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcbdd1f-a2f0-4003-b21a-8f300e7d9cb0 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:16.408599 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-6fcbdd1f-a2f0-4003-b21a-8f300e7d9cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:16.421764 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:16.441859 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e5769-c475-4876-aba3-5df4d673c997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:16.442838 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e5769-c475-4876-aba3-5df4d673c997 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:16.442948 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e5769-c475-4876-aba3-5df4d673c997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:16.443153 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e5769-c475-4876-aba3-5df4d673c997 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:16.443863 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e5769-c475-4876-aba3-5df4d673c997 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:16.445461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e5769-c475-4876-aba3-5df4d673c997 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:16.459144 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 956/1929] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:16.463523 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e5769-c475-4876-aba3-5df4d673c997 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:16.475645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e5769-c475-4876-aba3-5df4d673c997 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:16.477320 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcbdd1f-a2f0-4003-b21a-8f300e7d9cb0 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:16.480680 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858e79d-c96d-43a3-9377-ef0503dc279b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:16.481394 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858e79d-c96d-43a3-9377-ef0503dc279b admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:16.481919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858e79d-c96d-43a3-9377-ef0503dc279b admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:16.481994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14e662-121f-4197-8fc6-2d252879a3ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:16.482088 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858e79d-c96d-43a3-9377-ef0503dc279b admin admin] PATH_INFO: `/v3/projects/3c989e137c3b4803929fdb0433419c60/users/b3d2f631881e46b5bffaae677c1dad90/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:16.482450 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14e662-121f-4197-8fc6-2d252879a3ff service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:16.482917 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14e662-121f-4197-8fc6-2d252879a3ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:16.483356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14e662-121f-4197-8fc6-2d252879a3ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:16.484029 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858e79d-c96d-43a3-9377-ef0503dc279b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c989e137c3b4803929fdb0433419c60, user_id=b3d2f631881e46b5bffaae677c1dad90, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:16.487339 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858e79d-c96d-43a3-9377-ef0503dc279b admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:16.495452 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14e662-121f-4197-8fc6-2d252879a3ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:16.497032 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14e662-121f-4197-8fc6-2d252879a3ff service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:16.499339 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 957/1930] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:16 2022] PUT /identity/v3/projects/3c989e137c3b4803929fdb0433419c60/users/b3d2f631881e46b5bffaae677c1dad90/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:03:16.510300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 974/1931] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:03:16.511427 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada85f9-e35a-4a56-86a7-f14ae3c1caca None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:16.511788 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada85f9-e35a-4a56-86a7-f14ae3c1caca None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:16.512139 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada85f9-e35a-4a56-86a7-f14ae3c1caca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:16.542709 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-9ada85f9-e35a-4a56-86a7-f14ae3c1caca None None] MFA Rules not processed for user `b3d2f631881e46b5bffaae677c1dad90`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:16.544024 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-9ada85f9-e35a-4a56-86a7-f14ae3c1caca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:16.627307 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 958/1932] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:16 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:03:16.638690 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14e662-121f-4197-8fc6-2d252879a3ff service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:16.640277 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-4e14e662-121f-4197-8fc6-2d252879a3ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:16.712806 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14e662-121f-4197-8fc6-2d252879a3ff service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:16.761595 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8854486-08d5-47cd-9101-796a55ebd02f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:16.762220 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8854486-08d5-47cd-9101-796a55ebd02f service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:16.762499 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8854486-08d5-47cd-9101-796a55ebd02f service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:16.762775 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8854486-08d5-47cd-9101-796a55ebd02f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:16.772544 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8854486-08d5-47cd-9101-796a55ebd02f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:16.774013 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8854486-08d5-47cd-9101-796a55ebd02f service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:16.785201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 975/1933] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:16 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:03:16.806618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada85f9-e35a-4a56-86a7-f14ae3c1caca None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:16.808612 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-9ada85f9-e35a-4a56-86a7-f14ae3c1caca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:16.825605 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:16.853399 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8854486-08d5-47cd-9101-796a55ebd02f service cinder] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:16.854899 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-c8854486-08d5-47cd-9101-796a55ebd02f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:16.866635 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8854486-08d5-47cd-9101-796a55ebd02f service cinder] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:16.870394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eeb7e1-b538-4d09-a0c6-f597eeb7ab42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:16.871073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eeb7e1-b538-4d09-a0c6-f597eeb7ab42 service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:16.871421 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eeb7e1-b538-4d09-a0c6-f597eeb7ab42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:16.871736 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eeb7e1-b538-4d09-a0c6-f597eeb7ab42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:16.873475 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-02eeb7e1-b538-4d09-a0c6-f597eeb7ab42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:16.879595 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada85f9-e35a-4a56-86a7-f14ae3c1caca None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:16.884209 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25494ead-f9f1-4689-8e5f-e181dcdefd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:16.884898 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25494ead-f9f1-4689-8e5f-e181dcdefd41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:16.885191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25494ead-f9f1-4689-8e5f-e181dcdefd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:16.885452 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25494ead-f9f1-4689-8e5f-e181dcdefd41 admin admin] PATH_INFO: `/v3/projects/ac6fab5571ae4705827a24f15ba1b21e` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:16.887222 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25494ead-f9f1-4689-8e5f-e181dcdefd41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac6fab5571ae4705827a24f15ba1b21e)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:16.889191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25494ead-f9f1-4689-8e5f-e181dcdefd41 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:16.933859 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eeb7e1-b538-4d09-a0c6-f597eeb7ab42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:16.935318 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eeb7e1-b538-4d09-a0c6-f597eeb7ab42 service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:16.939452 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-25494ead-f9f1-4689-8e5f-e181dcdefd41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac6fab5571ae4705827a24f15ba1b21e', 'request_id': 'req-25494ead-f9f1-4689-8e5f-e181dcdefd41'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:16.944543 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 976/1934] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:16 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:03:16.947514 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-25494ead-f9f1-4689-8e5f-e181dcdefd41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac6fab5571ae4705827a24f15ba1b21e', 'request_id': 'req-25494ead-f9f1-4689-8e5f-e181dcdefd41'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:16.953305 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-25494ead-f9f1-4689-8e5f-e181dcdefd41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac6fab5571ae4705827a24f15ba1b21e', 'request_id': 'req-25494ead-f9f1-4689-8e5f-e181dcdefd41'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:16.966417 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-25494ead-f9f1-4689-8e5f-e181dcdefd41 admin admin] The token cache is being invalidate because project ac6fab5571ae4705827a24f15ba1b21e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:03:16.966706 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-25494ead-f9f1-4689-8e5f-e181dcdefd41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:16.967347 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-25494ead-f9f1-4689-8e5f-e181dcdefd41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:16.968875 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 959/1935] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:03:16 2022] DELETE /identity/v3/projects/ac6fab5571ae4705827a24f15ba1b21e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:16.973987 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eeb7e1-b538-4d09-a0c6-f597eeb7ab42 service cinder] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:16.985108 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eeb7e1-b538-4d09-a0c6-f597eeb7ab42 service cinder] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:16.992884 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:17.061733 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9c1aa-6da9-4708-8e60-cfd052da70d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:17.062402 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9c1aa-6da9-4708-8e60-cfd052da70d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:17.062714 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9c1aa-6da9-4708-8e60-cfd052da70d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:17.062989 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9c1aa-6da9-4708-8e60-cfd052da70d9 admin admin] PATH_INFO: `/v3/users/66546a7cf2ec49de9a24ae6953d40559` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:17.064206 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9c1aa-6da9-4708-8e60-cfd052da70d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66546a7cf2ec49de9a24ae6953d40559)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:17.066188 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9c1aa-6da9-4708-8e60-cfd052da70d9 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:17.128715 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-7bf9c1aa-6da9-4708-8e60-cfd052da70d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66546a7cf2ec49de9a24ae6953d40559'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:17.138266 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-7bf9c1aa-6da9-4708-8e60-cfd052da70d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66546a7cf2ec49de9a24ae6953d40559'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:17.171465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-7bf9c1aa-6da9-4708-8e60-cfd052da70d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66546a7cf2ec49de9a24ae6953d40559'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:17.209536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-7bf9c1aa-6da9-4708-8e60-cfd052da70d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66546a7cf2ec49de9a24ae6953d40559'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:17.210436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-7bf9c1aa-6da9-4708-8e60-cfd052da70d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66546a7cf2ec49de9a24ae6953d40559'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:17.220622 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 977/1936] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:03:16 2022] DELETE /identity/v3/users/66546a7cf2ec49de9a24ae6953d40559 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:17.440453 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25494ead-f9f1-4689-8e5f-e181dcdefd41 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:17.441986 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-25494ead-f9f1-4689-8e5f-e181dcdefd41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:17.462619 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25494ead-f9f1-4689-8e5f-e181dcdefd41 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:17.568675 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9f12b-f4ce-40f4-86b7-b6ebacbc97b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:17.569309 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9f12b-f4ce-40f4-86b7-b6ebacbc97b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:17.569596 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9f12b-f4ce-40f4-86b7-b6ebacbc97b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:17.569881 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9f12b-f4ce-40f4-86b7-b6ebacbc97b1 admin admin] PATH_INFO: `/v3/users/04bc57f4bc9042628e177c80fbf1e4b1` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:17.571782 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd9f12b-f4ce-40f4-86b7-b6ebacbc97b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04bc57f4bc9042628e177c80fbf1e4b1)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:17.574082 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd9f12b-f4ce-40f4-86b7-b6ebacbc97b1 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:17.582706 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9c1aa-6da9-4708-8e60-cfd052da70d9 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:17.585434 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-7bf9c1aa-6da9-4708-8e60-cfd052da70d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:17.605786 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9c1aa-6da9-4708-8e60-cfd052da70d9 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:17.614907 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92438939-730c-40c3-a845-5b9446aa39ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:17.615613 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92438939-730c-40c3-a845-5b9446aa39ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:17.616027 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92438939-730c-40c3-a845-5b9446aa39ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:17.616451 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92438939-730c-40c3-a845-5b9446aa39ca admin admin] PATH_INFO: `/v3/users/c3caca5de8034d8dab6f226ade43267f` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:17.618010 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92438939-730c-40c3-a845-5b9446aa39ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3caca5de8034d8dab6f226ade43267f)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:17.622399 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92438939-730c-40c3-a845-5b9446aa39ca admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:17.648336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:17.650882 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-7dd9f12b-f4ce-40f4-86b7-b6ebacbc97b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04bc57f4bc9042628e177c80fbf1e4b1'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:17.672781 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-7dd9f12b-f4ce-40f4-86b7-b6ebacbc97b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04bc57f4bc9042628e177c80fbf1e4b1'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:17.693633 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-7dd9f12b-f4ce-40f4-86b7-b6ebacbc97b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04bc57f4bc9042628e177c80fbf1e4b1'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:17.694930 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-7dd9f12b-f4ce-40f4-86b7-b6ebacbc97b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04bc57f4bc9042628e177c80fbf1e4b1'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:17.699365 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-92438939-730c-40c3-a845-5b9446aa39ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3caca5de8034d8dab6f226ade43267f'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:17.706295 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-7dd9f12b-f4ce-40f4-86b7-b6ebacbc97b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04bc57f4bc9042628e177c80fbf1e4b1'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:17.710815 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-92438939-730c-40c3-a845-5b9446aa39ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3caca5de8034d8dab6f226ade43267f'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:17.717883 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 960/1937] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:03:17 2022] DELETE /identity/v3/users/04bc57f4bc9042628e177c80fbf1e4b1 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:17.722731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-92438939-730c-40c3-a845-5b9446aa39ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3caca5de8034d8dab6f226ade43267f'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:17.727749 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa91660-bfd7-48f6-81dc-dbbf3765813e None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:17.728036 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa91660-bfd7-48f6-81dc-dbbf3765813e None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:17.728307 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa91660-bfd7-48f6-81dc-dbbf3765813e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:17.742310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-92438939-730c-40c3-a845-5b9446aa39ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3caca5de8034d8dab6f226ade43267f'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:17.743213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-92438939-730c-40c3-a845-5b9446aa39ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3caca5de8034d8dab6f226ade43267f'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:17.750349 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-4fa91660-bfd7-48f6-81dc-dbbf3765813e None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:17.751573 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-4fa91660-bfd7-48f6-81dc-dbbf3765813e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:17.754944 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 978/1938] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:03:17 2022] DELETE /identity/v3/users/c3caca5de8034d8dab6f226ade43267f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:03:17.756554 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2829f-a2a9-4232-91ce-19953864f70a None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:17.756874 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2829f-a2a9-4232-91ce-19953864f70a None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:17.757149 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2829f-a2a9-4232-91ce-19953864f70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:17.769147 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:17.789712 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-ddd2829f-a2a9-4232-91ce-19953864f70a None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:17.791372 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-ddd2829f-a2a9-4232-91ce-19953864f70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:17.829866 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 961/1939] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:03:17.834080 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa91660-bfd7-48f6-81dc-dbbf3765813e None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:17.835433 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-4fa91660-bfd7-48f6-81dc-dbbf3765813e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:17.847412 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa91660-bfd7-48f6-81dc-dbbf3765813e None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:17.852020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c2e05-d457-4365-b3b6-24f4d14098dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:17.852675 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c2e05-d457-4365-b3b6-24f4d14098dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:17.852974 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c2e05-d457-4365-b3b6-24f4d14098dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:17.853234 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c2e05-d457-4365-b3b6-24f4d14098dd admin admin] PATH_INFO: `/v3/projects/5fab7c5df62b4a61bc09e491b27ae713` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:17.854380 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c2e05-d457-4365-b3b6-24f4d14098dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fab7c5df62b4a61bc09e491b27ae713)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:17.855786 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 979/1940] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:17.856260 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c2e05-d457-4365-b3b6-24f4d14098dd admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:17.856931 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2829f-a2a9-4232-91ce-19953864f70a None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:17.858191 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-ddd2829f-a2a9-4232-91ce-19953864f70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:17.905476 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-291c2e05-d457-4365-b3b6-24f4d14098dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fab7c5df62b4a61bc09e491b27ae713', 'request_id': 'req-291c2e05-d457-4365-b3b6-24f4d14098dd'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:17.913881 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-291c2e05-d457-4365-b3b6-24f4d14098dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fab7c5df62b4a61bc09e491b27ae713', 'request_id': 'req-291c2e05-d457-4365-b3b6-24f4d14098dd'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:17.919941 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-291c2e05-d457-4365-b3b6-24f4d14098dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fab7c5df62b4a61bc09e491b27ae713', 'request_id': 'req-291c2e05-d457-4365-b3b6-24f4d14098dd'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:17.925829 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2829f-a2a9-4232-91ce-19953864f70a None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:17.931513 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-291c2e05-d457-4365-b3b6-24f4d14098dd admin admin] The token cache is being invalidate because project 5fab7c5df62b4a61bc09e491b27ae713 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:03:17.931806 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-291c2e05-d457-4365-b3b6-24f4d14098dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:17.932477 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-291c2e05-d457-4365-b3b6-24f4d14098dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:17.934131 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 962/1941] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:03:17 2022] DELETE /identity/v3/projects/5fab7c5df62b4a61bc09e491b27ae713 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:17.934949 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c2e05-d457-4365-b3b6-24f4d14098dd admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:17.936214 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-291c2e05-d457-4365-b3b6-24f4d14098dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:17.979667 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0825b2eb-9c59-4437-8d71-6b2d7ba11efd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:17.980291 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825b2eb-9c59-4437-8d71-6b2d7ba11efd service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:17.980588 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825b2eb-9c59-4437-8d71-6b2d7ba11efd service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:17.980876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825b2eb-9c59-4437-8d71-6b2d7ba11efd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:17.982771 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-0825b2eb-9c59-4437-8d71-6b2d7ba11efd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:17.989270 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:17.994486 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c2e05-d457-4365-b3b6-24f4d14098dd admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:18.008432 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322a851b-542b-4022-aea9-48a431ce8180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:18.009446 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a851b-542b-4022-aea9-48a431ce8180 service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:18.009788 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a851b-542b-4022-aea9-48a431ce8180 service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:18.010069 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a851b-542b-4022-aea9-48a431ce8180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:18.012465 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-322a851b-542b-4022-aea9-48a431ce8180 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:18.056870 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0825b2eb-9c59-4437-8d71-6b2d7ba11efd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:18.058361 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0825b2eb-9c59-4437-8d71-6b2d7ba11efd service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:18.074197 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 980/1942] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:03:18.077795 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155e531-82f4-4222-ae90-8e5081ea095a None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:18.078405 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155e531-82f4-4222-ae90-8e5081ea095a None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:18.078973 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155e531-82f4-4222-ae90-8e5081ea095a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:18.090885 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322a851b-542b-4022-aea9-48a431ce8180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:18.092643 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322a851b-542b-4022-aea9-48a431ce8180 service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:18.105242 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 963/1943] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:03:18.107846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-9155e531-82f4-4222-ae90-8e5081ea095a None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:18.109286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-9155e531-82f4-4222-ae90-8e5081ea095a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:18.119082 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 981/1944] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:18.127577 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e552b3-9f21-4b0b-8d64-1314a464bc61 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:18.128670 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e552b3-9f21-4b0b-8d64-1314a464bc61 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:18.129253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e552b3-9f21-4b0b-8d64-1314a464bc61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:18.154925 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-f7e552b3-9f21-4b0b-8d64-1314a464bc61 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:18.155688 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-f7e552b3-9f21-4b0b-8d64-1314a464bc61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:18.165148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 964/1945] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:03:18.169325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155e531-82f4-4222-ae90-8e5081ea095a None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:18.175554 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:18.186775 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155e531-82f4-4222-ae90-8e5081ea095a None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:18.192845 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af86c9-c8e7-46b6-b62d-91b5227bf5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:18.193478 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af86c9-c8e7-46b6-b62d-91b5227bf5b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:18.193760 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af86c9-c8e7-46b6-b62d-91b5227bf5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:18.194044 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af86c9-c8e7-46b6-b62d-91b5227bf5b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:18.194740 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af86c9-c8e7-46b6-b62d-91b5227bf5b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:03:18.195047 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af86c9-c8e7-46b6-b62d-91b5227bf5b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:18.196706 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af86c9-c8e7-46b6-b62d-91b5227bf5b0 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:18.211214 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 982/1946] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:03:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:03:18.215598 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e552b3-9f21-4b0b-8d64-1314a464bc61 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:18.226288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e552b3-9f21-4b0b-8d64-1314a464bc61 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:18.231856 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb0889-0d96-447d-8c87-06d3b1c0afb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:18.232704 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb0889-0d96-447d-8c87-06d3b1c0afb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:18.232991 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb0889-0d96-447d-8c87-06d3b1c0afb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:18.233325 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb0889-0d96-447d-8c87-06d3b1c0afb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:18.234054 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb0889-0d96-447d-8c87-06d3b1c0afb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:18.235917 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb0889-0d96-447d-8c87-06d3b1c0afb5 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:18.272689 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 965/1947] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:03:18 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:03:18.274501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af86c9-c8e7-46b6-b62d-91b5227bf5b0 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:18.285423 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af86c9-c8e7-46b6-b62d-91b5227bf5b0 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:18.345622 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cdbe28-30ca-4e85-b0e9-e8214c326708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:18.347110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cdbe28-30ca-4e85-b0e9-e8214c326708 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:18.347484 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cdbe28-30ca-4e85-b0e9-e8214c326708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:18.347734 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cdbe28-30ca-4e85-b0e9-e8214c326708 admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:18.348534 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cdbe28-30ca-4e85-b0e9-e8214c326708 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:18.350706 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cdbe28-30ca-4e85-b0e9-e8214c326708 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:18.379775 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 983/1948] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:03:18 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:18.382246 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb0889-0d96-447d-8c87-06d3b1c0afb5 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:18.399331 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb0889-0d96-447d-8c87-06d3b1c0afb5 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:18.406431 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd01fca0-7cfb-418b-917a-98fdeb84a077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:18.407478 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01fca0-7cfb-418b-917a-98fdeb84a077 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:18.407478 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01fca0-7cfb-418b-917a-98fdeb84a077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:18.407924 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01fca0-7cfb-418b-917a-98fdeb84a077 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:18.409446 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01fca0-7cfb-418b-917a-98fdeb84a077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:18.411453 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01fca0-7cfb-418b-917a-98fdeb84a077 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:18.425656 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 966/1949] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:18.429711 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cdbe28-30ca-4e85-b0e9-e8214c326708 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:18.444377 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cdbe28-30ca-4e85-b0e9-e8214c326708 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:18.458155 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92584144-014b-4b26-b0db-a358c443e706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:18.458794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92584144-014b-4b26-b0db-a358c443e706 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:18.459077 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92584144-014b-4b26-b0db-a358c443e706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:18.459350 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92584144-014b-4b26-b0db-a358c443e706 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:18.460052 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92584144-014b-4b26-b0db-a358c443e706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:18.461884 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92584144-014b-4b26-b0db-a358c443e706 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:18.479834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 984/1950] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:18.484236 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd01fca0-7cfb-418b-917a-98fdeb84a077 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:18.495509 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd01fca0-7cfb-418b-917a-98fdeb84a077 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:18.500611 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c5ae60-a419-49b0-823c-d5b0b729e81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:18.501449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5ae60-a419-49b0-823c-d5b0b729e81f admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:18.501734 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5ae60-a419-49b0-823c-d5b0b729e81f admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:18.502008 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5ae60-a419-49b0-823c-d5b0b729e81f admin admin] PATH_INFO: `/v3/projects/44794c3479904067b96e6519a8118f6c/users/345aa188de3b49f5b5205983764e152e/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:18.520863 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c5ae60-a419-49b0-823c-d5b0b729e81f admin admin] RBAC: Authorizing `identity:create_grant(project_id=44794c3479904067b96e6519a8118f6c, user_id=345aa188de3b49f5b5205983764e152e, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:18.523753 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c5ae60-a419-49b0-823c-d5b0b729e81f admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:18.543932 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 967/1951] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:18 2022] PUT /identity/v3/projects/44794c3479904067b96e6519a8118f6c/users/345aa188de3b49f5b5205983764e152e/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:18.555418 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe8918-c419-4bde-a488-7afaf3c276ab None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:18.555701 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe8918-c419-4bde-a488-7afaf3c276ab None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:18.555983 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe8918-c419-4bde-a488-7afaf3c276ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:18.592613 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-cafe8918-c419-4bde-a488-7afaf3c276ab None None] MFA Rules not processed for user `345aa188de3b49f5b5205983764e152e`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:18.593847 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-cafe8918-c419-4bde-a488-7afaf3c276ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:18.717922 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 985/1952] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:18.718332 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c5ae60-a419-49b0-823c-d5b0b729e81f admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:18.821499 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c5ae60-a419-49b0-823c-d5b0b729e81f admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:18.826471 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5c741-6326-48a5-8a6e-c2210d4b8545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:18.827107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5c741-6326-48a5-8a6e-c2210d4b8545 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:18.827539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5c741-6326-48a5-8a6e-c2210d4b8545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:18.827959 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5c741-6326-48a5-8a6e-c2210d4b8545 admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:18.828688 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a5c741-6326-48a5-8a6e-c2210d4b8545 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:18.831371 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a5c741-6326-48a5-8a6e-c2210d4b8545 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:18.885962 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 968/1953] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:03:18 2022] POST /identity/v3/projects => generated 385 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:03:18.888158 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe8918-c419-4bde-a488-7afaf3c276ab None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:18.906875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe8918-c419-4bde-a488-7afaf3c276ab None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:18.995308 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:19.014575 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97fc8a3-bab9-45d1-9716-5f0b2dd6f793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:19.015995 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97fc8a3-bab9-45d1-9716-5f0b2dd6f793 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:19.016532 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97fc8a3-bab9-45d1-9716-5f0b2dd6f793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:19.017119 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97fc8a3-bab9-45d1-9716-5f0b2dd6f793 admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:19.018424 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97fc8a3-bab9-45d1-9716-5f0b2dd6f793 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:19.022159 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97fc8a3-bab9-45d1-9716-5f0b2dd6f793 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:19.066011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 986/1954] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:03:18 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:19.067710 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5c741-6326-48a5-8a6e-c2210d4b8545 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:19.078348 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5c741-6326-48a5-8a6e-c2210d4b8545 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:19.084631 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f55818-94d8-4e6a-bea7-601b2800b71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:19.085593 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f55818-94d8-4e6a-bea7-601b2800b71e admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:19.086077 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f55818-94d8-4e6a-bea7-601b2800b71e admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:19.086357 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f55818-94d8-4e6a-bea7-601b2800b71e admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:19.087110 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f55818-94d8-4e6a-bea7-601b2800b71e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:19.088724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f55818-94d8-4e6a-bea7-601b2800b71e admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:19.100396 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 969/1955] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:03:19.105649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97fc8a3-bab9-45d1-9716-5f0b2dd6f793 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:19.120190 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97fc8a3-bab9-45d1-9716-5f0b2dd6f793 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:19.127197 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d61455-fc38-4a33-8413-40b82fe646f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:19.128141 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d61455-fc38-4a33-8413-40b82fe646f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:19.128397 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d61455-fc38-4a33-8413-40b82fe646f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:19.128749 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d61455-fc38-4a33-8413-40b82fe646f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:19.130219 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d61455-fc38-4a33-8413-40b82fe646f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:19.131122 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d61455-fc38-4a33-8413-40b82fe646f6 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:19.149913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 987/1956] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:19.151786 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f55818-94d8-4e6a-bea7-601b2800b71e admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:19.178325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d61455-fc38-4a33-8413-40b82fe646f6 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:19.180271 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-b6d61455-fc38-4a33-8413-40b82fe646f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:19.183513 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f55818-94d8-4e6a-bea7-601b2800b71e admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:19.197313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d61455-fc38-4a33-8413-40b82fe646f6 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:19.202493 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d9dbf-0d43-4af5-a67b-e08c3291213b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:19.203184 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d9dbf-0d43-4af5-a67b-e08c3291213b admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:19.203465 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d9dbf-0d43-4af5-a67b-e08c3291213b admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:19.203744 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d9dbf-0d43-4af5-a67b-e08c3291213b admin admin] PATH_INFO: `/v3/projects/189b4885ee8147d599f396c1bcd07f1d/users/caa1532321c146b39e9b99e16c62772d/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:19.207165 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:19.210225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0de2e4-fe33-4a22-af6b-6866b468ef62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:19.221936 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0de2e4-fe33-4a22-af6b-6866b468ef62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:19.224987 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0de2e4-fe33-4a22-af6b-6866b468ef62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:19.225988 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0de2e4-fe33-4a22-af6b-6866b468ef62 admin admin] PATH_INFO: `/v3/projects/be9db04a0d554c45989492d020ec279f` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:19.227389 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0de2e4-fe33-4a22-af6b-6866b468ef62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be9db04a0d554c45989492d020ec279f)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:19.228893 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d9dbf-0d43-4af5-a67b-e08c3291213b admin admin] RBAC: Authorizing `identity:create_grant(project_id=189b4885ee8147d599f396c1bcd07f1d, user_id=caa1532321c146b39e9b99e16c62772d, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:19.230635 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0de2e4-fe33-4a22-af6b-6866b468ef62 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:19.232486 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d9dbf-0d43-4af5-a67b-e08c3291213b admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:19.248689 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 970/1957] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:19 2022] PUT /identity/v3/projects/189b4885ee8147d599f396c1bcd07f1d/users/caa1532321c146b39e9b99e16c62772d/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:19.249877 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d9dbf-0d43-4af5-a67b-e08c3291213b admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:19.260079 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-d71d9dbf-0d43-4af5-a67b-e08c3291213b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:19.291228 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-8b0de2e4-fe33-4a22-af6b-6866b468ef62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be9db04a0d554c45989492d020ec279f', 'request_id': 'req-8b0de2e4-fe33-4a22-af6b-6866b468ef62'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:19.306851 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-8b0de2e4-fe33-4a22-af6b-6866b468ef62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be9db04a0d554c45989492d020ec279f', 'request_id': 'req-8b0de2e4-fe33-4a22-af6b-6866b468ef62'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:19.318851 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-8b0de2e4-fe33-4a22-af6b-6866b468ef62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be9db04a0d554c45989492d020ec279f', 'request_id': 'req-8b0de2e4-fe33-4a22-af6b-6866b468ef62'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:19.332678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d9dbf-0d43-4af5-a67b-e08c3291213b admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:19.334801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-8b0de2e4-fe33-4a22-af6b-6866b468ef62 admin admin] The token cache is being invalidate because project be9db04a0d554c45989492d020ec279f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:03:19.335235 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-8b0de2e4-fe33-4a22-af6b-6866b468ef62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:19.335959 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-8b0de2e4-fe33-4a22-af6b-6866b468ef62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:19.340974 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 988/1958] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:03:19 2022] DELETE /identity/v3/projects/be9db04a0d554c45989492d020ec279f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:03:19.340974 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed41d2-409c-4a59-b419-f94a429ea506 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:19.340974 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed41d2-409c-4a59-b419-f94a429ea506 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:19.340974 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed41d2-409c-4a59-b419-f94a429ea506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:19.369255 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-5bed41d2-409c-4a59-b419-f94a429ea506 None None] MFA Rules not processed for user `caa1532321c146b39e9b99e16c62772d`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:19.370528 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5bed41d2-409c-4a59-b419-f94a429ea506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:19.412156 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f522378-f58d-45a6-8582-8b26a62440a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:19.412842 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f522378-f58d-45a6-8582-8b26a62440a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:19.413122 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f522378-f58d-45a6-8582-8b26a62440a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:19.413414 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f522378-f58d-45a6-8582-8b26a62440a5 admin admin] PATH_INFO: `/v3/projects/4ae21d4c475b4f56896aef81c29d4a30` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:19.414586 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f522378-f58d-45a6-8582-8b26a62440a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ae21d4c475b4f56896aef81c29d4a30)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:19.416481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f522378-f58d-45a6-8582-8b26a62440a5 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:19.451578 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 989/1959] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:19.451578 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bed41d2-409c-4a59-b419-f94a429ea506 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:19.451578 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5bed41d2-409c-4a59-b419-f94a429ea506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:19.466415 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bed41d2-409c-4a59-b419-f94a429ea506 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:19.476092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-7f522378-f58d-45a6-8582-8b26a62440a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ae21d4c475b4f56896aef81c29d4a30', 'request_id': 'req-7f522378-f58d-45a6-8582-8b26a62440a5'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:19.484970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-7f522378-f58d-45a6-8582-8b26a62440a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ae21d4c475b4f56896aef81c29d4a30', 'request_id': 'req-7f522378-f58d-45a6-8582-8b26a62440a5'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:19.492081 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-7f522378-f58d-45a6-8582-8b26a62440a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ae21d4c475b4f56896aef81c29d4a30', 'request_id': 'req-7f522378-f58d-45a6-8582-8b26a62440a5'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:19.505710 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-7f522378-f58d-45a6-8582-8b26a62440a5 admin admin] The token cache is being invalidate because project 4ae21d4c475b4f56896aef81c29d4a30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:03:19.506066 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-7f522378-f58d-45a6-8582-8b26a62440a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:19.506859 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-7f522378-f58d-45a6-8582-8b26a62440a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:19.508401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 971/1960] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:03:19 2022] DELETE /identity/v3/projects/4ae21d4c475b4f56896aef81c29d4a30 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:19.509170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f522378-f58d-45a6-8582-8b26a62440a5 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:19.512469 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-7f522378-f58d-45a6-8582-8b26a62440a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:19.521591 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:19.536210 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:19.536646 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aca16f-c08d-4eb7-a97c-3c32015bac88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:19.537617 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aca16f-c08d-4eb7-a97c-3c32015bac88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:19.537977 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aca16f-c08d-4eb7-a97c-3c32015bac88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:19.538364 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aca16f-c08d-4eb7-a97c-3c32015bac88 admin admin] PATH_INFO: `/v3/users/d085c6d872e94543b2bfd3e19311bf20` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:19.539954 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aca16f-c08d-4eb7-a97c-3c32015bac88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d085c6d872e94543b2bfd3e19311bf20)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:19.541950 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aca16f-c08d-4eb7-a97c-3c32015bac88 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:19.585358 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f522378-f58d-45a6-8582-8b26a62440a5 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:19.590052 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-a4aca16f-c08d-4eb7-a97c-3c32015bac88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd085c6d872e94543b2bfd3e19311bf20'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:19.590170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d671a99-f2db-4ddc-8bf7-9b9d086dc69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:19.590790 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d671a99-f2db-4ddc-8bf7-9b9d086dc69a admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:19.591140 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d671a99-f2db-4ddc-8bf7-9b9d086dc69a admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:19.591614 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d671a99-f2db-4ddc-8bf7-9b9d086dc69a admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:19.592212 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d671a99-f2db-4ddc-8bf7-9b9d086dc69a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:19.594230 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d671a99-f2db-4ddc-8bf7-9b9d086dc69a admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:19.597880 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-a4aca16f-c08d-4eb7-a97c-3c32015bac88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd085c6d872e94543b2bfd3e19311bf20'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:19.605247 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-a4aca16f-c08d-4eb7-a97c-3c32015bac88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd085c6d872e94543b2bfd3e19311bf20'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:19.619149 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-a4aca16f-c08d-4eb7-a97c-3c32015bac88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd085c6d872e94543b2bfd3e19311bf20'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:19.619865 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-a4aca16f-c08d-4eb7-a97c-3c32015bac88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd085c6d872e94543b2bfd3e19311bf20'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:19.623639 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 972/1961] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:03:19 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:19.625898 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 990/1962] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:03:19 2022] DELETE /identity/v3/users/d085c6d872e94543b2bfd3e19311bf20 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:19.629646 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d671a99-f2db-4ddc-8bf7-9b9d086dc69a admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:19.629752 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aca16f-c08d-4eb7-a97c-3c32015bac88 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:19.629914 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-4d671a99-f2db-4ddc-8bf7-9b9d086dc69a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:19.631066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-a4aca16f-c08d-4eb7-a97c-3c32015bac88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:19.713616 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aca16f-c08d-4eb7-a97c-3c32015bac88 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:19.721805 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d6cf4-c753-485a-8f3d-1495ae387b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:19.722617 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d6cf4-c753-485a-8f3d-1495ae387b7e admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:19.722902 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d6cf4-c753-485a-8f3d-1495ae387b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:19.723182 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d6cf4-c753-485a-8f3d-1495ae387b7e admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:19.723859 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d6cf4-c753-485a-8f3d-1495ae387b7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:19.724303 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d671a99-f2db-4ddc-8bf7-9b9d086dc69a admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:19.726125 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d6cf4-c753-485a-8f3d-1495ae387b7e admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:19.730786 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb34df-749e-4d65-9d65-0fb030a9b4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:19.731558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb34df-749e-4d65-9d65-0fb030a9b4cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:19.731944 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb34df-749e-4d65-9d65-0fb030a9b4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:19.732887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb34df-749e-4d65-9d65-0fb030a9b4cf admin admin] PATH_INFO: `/v3/users/faeb7733ef744a3392c2339c570515eb` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:19.734259 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fb34df-749e-4d65-9d65-0fb030a9b4cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=faeb7733ef744a3392c2339c570515eb)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:19.736299 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fb34df-749e-4d65-9d65-0fb030a9b4cf admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:19.750600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 991/1963] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:03:19 2022] POST /identity/v3/users => generated 299 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:19.758745 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d6cf4-c753-485a-8f3d-1495ae387b7e admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:19.774946 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d6cf4-c753-485a-8f3d-1495ae387b7e admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:19.782238 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a023d663-77f5-474e-b2fa-6cb003123365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:19.783114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023d663-77f5-474e-b2fa-6cb003123365 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:19.783497 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023d663-77f5-474e-b2fa-6cb003123365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:19.783991 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023d663-77f5-474e-b2fa-6cb003123365 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:19.786182 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a023d663-77f5-474e-b2fa-6cb003123365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:19.788798 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a023d663-77f5-474e-b2fa-6cb003123365 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:19.803683 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 992/1964] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:19.804940 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-38fb34df-749e-4d65-9d65-0fb030a9b4cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faeb7733ef744a3392c2339c570515eb'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:19.819862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a023d663-77f5-474e-b2fa-6cb003123365 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:19.837123 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-38fb34df-749e-4d65-9d65-0fb030a9b4cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faeb7733ef744a3392c2339c570515eb'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:19.845849 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a023d663-77f5-474e-b2fa-6cb003123365 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:19.856955 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf4e881-9c5c-4ecc-bc2b-ed07d42b5807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:19.858608 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf4e881-9c5c-4ecc-bc2b-ed07d42b5807 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:19.858894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-38fb34df-749e-4d65-9d65-0fb030a9b4cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faeb7733ef744a3392c2339c570515eb'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:19.859477 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf4e881-9c5c-4ecc-bc2b-ed07d42b5807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:19.859732 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-38fb34df-749e-4d65-9d65-0fb030a9b4cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faeb7733ef744a3392c2339c570515eb'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:19.860691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf4e881-9c5c-4ecc-bc2b-ed07d42b5807 admin admin] PATH_INFO: `/v3/projects/1f888c06fd33447492598cdb37c980eb/users/84b52d77e8b14b92b7d1e6791a9d082a/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:19.869387 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-38fb34df-749e-4d65-9d65-0fb030a9b4cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faeb7733ef744a3392c2339c570515eb'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:19.876149 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 973/1965] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:03:19 2022] DELETE /identity/v3/users/faeb7733ef744a3392c2339c570515eb => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:19.885089 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf4e881-9c5c-4ecc-bc2b-ed07d42b5807 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f888c06fd33447492598cdb37c980eb, user_id=84b52d77e8b14b92b7d1e6791a9d082a, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:19.887835 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf4e881-9c5c-4ecc-bc2b-ed07d42b5807 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:19.906723 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 993/1966] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:19 2022] PUT /identity/v3/projects/1f888c06fd33447492598cdb37c980eb/users/84b52d77e8b14b92b7d1e6791a9d082a/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:19.912391 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb34df-749e-4d65-9d65-0fb030a9b4cf admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:19.913917 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-38fb34df-749e-4d65-9d65-0fb030a9b4cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:19.949063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:20.006408 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb34df-749e-4d65-9d65-0fb030a9b4cf admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:20.009876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf4e881-9c5c-4ecc-bc2b-ed07d42b5807 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:20.012114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-2bf4e881-9c5c-4ecc-bc2b-ed07d42b5807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:20.054543 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:20.121804 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf4e881-9c5c-4ecc-bc2b-ed07d42b5807 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:20.128826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa0c24-fb38-49df-b531-2f6e0f7cec69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:20.130580 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa0c24-fb38-49df-b531-2f6e0f7cec69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:20.130937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa0c24-fb38-49df-b531-2f6e0f7cec69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:20.131337 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa0c24-fb38-49df-b531-2f6e0f7cec69 admin admin] PATH_INFO: `/v3/projects/19a89aa5894d4b0f8034d04bdc4494d0` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:20.132426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54f6a54-95ce-40cb-b06c-85a282200486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:20.132958 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affa0c24-fb38-49df-b531-2f6e0f7cec69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19a89aa5894d4b0f8034d04bdc4494d0)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:20.133882 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f6a54-95ce-40cb-b06c-85a282200486 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:20.134756 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f6a54-95ce-40cb-b06c-85a282200486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:20.134914 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affa0c24-fb38-49df-b531-2f6e0f7cec69 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:20.135755 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f6a54-95ce-40cb-b06c-85a282200486 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:20.137569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54f6a54-95ce-40cb-b06c-85a282200486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:20.140495 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54f6a54-95ce-40cb-b06c-85a282200486 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:20.159693 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 974/1967] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:20.164858 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54f6a54-95ce-40cb-b06c-85a282200486 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:20.179948 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54f6a54-95ce-40cb-b06c-85a282200486 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:20.203117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb9a64-3700-466a-b3d1-27464c586a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:20.203758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb9a64-3700-466a-b3d1-27464c586a96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:20.204805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb9a64-3700-466a-b3d1-27464c586a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:20.205006 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb9a64-3700-466a-b3d1-27464c586a96 admin admin] PATH_INFO: `/v3/projects/1f888c06fd33447492598cdb37c980eb/users/84b52d77e8b14b92b7d1e6791a9d082a/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:20.209488 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb9a64-3700-466a-b3d1-27464c586a96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f888c06fd33447492598cdb37c980eb, user_id=84b52d77e8b14b92b7d1e6791a9d082a, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:20.214538 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb9a64-3700-466a-b3d1-27464c586a96 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:20.221600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-affa0c24-fb38-49df-b531-2f6e0f7cec69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19a89aa5894d4b0f8034d04bdc4494d0', 'request_id': 'req-affa0c24-fb38-49df-b531-2f6e0f7cec69'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:20.230194 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-affa0c24-fb38-49df-b531-2f6e0f7cec69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19a89aa5894d4b0f8034d04bdc4494d0', 'request_id': 'req-affa0c24-fb38-49df-b531-2f6e0f7cec69'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:20.230706 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 975/1968] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:20 2022] PUT /identity/v3/projects/1f888c06fd33447492598cdb37c980eb/users/84b52d77e8b14b92b7d1e6791a9d082a/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:20.241140 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3987791-cf7b-4796-b92a-af5e2c9bda9f None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:20.242177 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3987791-cf7b-4796-b92a-af5e2c9bda9f None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:20.242744 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3987791-cf7b-4796-b92a-af5e2c9bda9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:20.252208 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-affa0c24-fb38-49df-b531-2f6e0f7cec69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19a89aa5894d4b0f8034d04bdc4494d0', 'request_id': 'req-affa0c24-fb38-49df-b531-2f6e0f7cec69'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:20.267331 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-affa0c24-fb38-49df-b531-2f6e0f7cec69 admin admin] The token cache is being invalidate because project 19a89aa5894d4b0f8034d04bdc4494d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:03:20.267717 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-affa0c24-fb38-49df-b531-2f6e0f7cec69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:20.268402 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-affa0c24-fb38-49df-b531-2f6e0f7cec69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:20.270112 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 994/1969] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:03:20 2022] DELETE /identity/v3/projects/19a89aa5894d4b0f8034d04bdc4494d0 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:20.280557 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-d3987791-cf7b-4796-b92a-af5e2c9bda9f None None] MFA Rules not processed for user `84b52d77e8b14b92b7d1e6791a9d082a`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:20.281803 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-d3987791-cf7b-4796-b92a-af5e2c9bda9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:20.287227 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6ec5e-536b-4a0d-95ef-fffd026418b4 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:20.287619 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6ec5e-536b-4a0d-95ef-fffd026418b4 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:20.288346 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6ec5e-536b-4a0d-95ef-fffd026418b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:20.311647 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-19f6ec5e-536b-4a0d-95ef-fffd026418b4 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:20.313027 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-19f6ec5e-536b-4a0d-95ef-fffd026418b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:20.364777 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 976/1970] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:20 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:03:20.365897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3987791-cf7b-4796-b92a-af5e2c9bda9f None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:20.367190 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-d3987791-cf7b-4796-b92a-af5e2c9bda9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:20.381548 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 995/1971] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:20.382387 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f6ec5e-536b-4a0d-95ef-fffd026418b4 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:20.383695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-19f6ec5e-536b-4a0d-95ef-fffd026418b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:20.416497 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:20.440094 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3987791-cf7b-4796-b92a-af5e2c9bda9f None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:20.441582 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f6ec5e-536b-4a0d-95ef-fffd026418b4 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:20.444708 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c605a1-5389-4f88-90c4-07980aed636f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:20.445172 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002fda70-66a0-4e70-ad25-48c2aead81c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:20.445452 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c605a1-5389-4f88-90c4-07980aed636f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:20.445732 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c605a1-5389-4f88-90c4-07980aed636f admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:20.445822 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fda70-66a0-4e70-ad25-48c2aead81c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:20.446000 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c605a1-5389-4f88-90c4-07980aed636f admin admin] PATH_INFO: `/v3/projects/30817664bd81428ca8cbbe436257838e` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:20.446097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fda70-66a0-4e70-ad25-48c2aead81c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:20.446304 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fda70-66a0-4e70-ad25-48c2aead81c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:20.447135 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c605a1-5389-4f88-90c4-07980aed636f admin admin] RBAC: Authorizing `identity:delete_project(project_id=30817664bd81428ca8cbbe436257838e)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:20.448036 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-002fda70-66a0-4e70-ad25-48c2aead81c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:20.449092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c605a1-5389-4f88-90c4-07980aed636f admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:20.456373 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:20.497093 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-b8c605a1-5389-4f88-90c4-07980aed636f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30817664bd81428ca8cbbe436257838e', 'request_id': 'req-b8c605a1-5389-4f88-90c4-07980aed636f'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:20.506097 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-b8c605a1-5389-4f88-90c4-07980aed636f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30817664bd81428ca8cbbe436257838e', 'request_id': 'req-b8c605a1-5389-4f88-90c4-07980aed636f'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:20.512101 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-b8c605a1-5389-4f88-90c4-07980aed636f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30817664bd81428ca8cbbe436257838e', 'request_id': 'req-b8c605a1-5389-4f88-90c4-07980aed636f'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:20.513579 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002fda70-66a0-4e70-ad25-48c2aead81c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:20.515029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002fda70-66a0-4e70-ad25-48c2aead81c8 service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:20.523544 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-b8c605a1-5389-4f88-90c4-07980aed636f admin admin] The token cache is being invalidate because project 30817664bd81428ca8cbbe436257838e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:03:20.523828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-b8c605a1-5389-4f88-90c4-07980aed636f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:20.524293 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 996/1972] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:03:20.524542 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-b8c605a1-5389-4f88-90c4-07980aed636f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:20.525550 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfbbabb-b62e-401d-9bf0-3d53ff9c7089 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:20.525839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfbbabb-b62e-401d-9bf0-3d53ff9c7089 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:20.526119 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfbbabb-b62e-401d-9bf0-3d53ff9c7089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:20.526232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 977/1973] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:03:20 2022] DELETE /identity/v3/projects/30817664bd81428ca8cbbe436257838e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:20.531119 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17dfc9-104b-4315-af0d-620c2bda3d48 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:20.531410 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17dfc9-104b-4315-af0d-620c2bda3d48 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:20.531691 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17dfc9-104b-4315-af0d-620c2bda3d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:20.551818 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-7dfbbabb-b62e-401d-9bf0-3d53ff9c7089 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:20.553101 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-1a17dfc9-104b-4315-af0d-620c2bda3d48 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:20.553676 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-7dfbbabb-b62e-401d-9bf0-3d53ff9c7089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:20.554276 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-1a17dfc9-104b-4315-af0d-620c2bda3d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:20.624839 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 978/1974] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:20.633146 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6df444-1493-4fc4-a817-4b79fa01572d None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:20.633381 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6df444-1493-4fc4-a817-4b79fa01572d None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:20.633750 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6df444-1493-4fc4-a817-4b79fa01572d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:20.639700 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 997/1975] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:20.644368 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfbbabb-b62e-401d-9bf0-3d53ff9c7089 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:20.655447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfbbabb-b62e-401d-9bf0-3d53ff9c7089 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:20.660358 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-ac6df444-1493-4fc4-a817-4b79fa01572d None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:20.661209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc66996d-f944-4237-824c-be226f825ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:20.661857 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66996d-f944-4237-824c-be226f825ae6 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:20.662137 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66996d-f944-4237-824c-be226f825ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:20.662247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-ac6df444-1493-4fc4-a817-4b79fa01572d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:20.662426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66996d-f944-4237-824c-be226f825ae6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:20.663100 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc66996d-f944-4237-824c-be226f825ae6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:03:20.663401 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc66996d-f944-4237-824c-be226f825ae6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:20.665265 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc66996d-f944-4237-824c-be226f825ae6 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:20.672610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 979/1976] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:03:20.676878 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6df444-1493-4fc4-a817-4b79fa01572d None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:20.680616 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 998/1977] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:03:20.682308 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc66996d-f944-4237-824c-be226f825ae6 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:20.691867 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6df444-1493-4fc4-a817-4b79fa01572d None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:20.692948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc66996d-f944-4237-824c-be226f825ae6 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:20.697756 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a37416-89d3-4bc1-b60f-62cfdd82517f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:20.698406 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a37416-89d3-4bc1-b60f-62cfdd82517f admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:20.698669 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e658d-153b-4694-af20-59d9220d482d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:20.698783 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a37416-89d3-4bc1-b60f-62cfdd82517f admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:20.699061 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a37416-89d3-4bc1-b60f-62cfdd82517f admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:20.699671 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e658d-153b-4694-af20-59d9220d482d admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:20.699797 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e658d-153b-4694-af20-59d9220d482d admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:20.699878 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a37416-89d3-4bc1-b60f-62cfdd82517f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:03:20.700005 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e658d-153b-4694-af20-59d9220d482d admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:20.700144 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a37416-89d3-4bc1-b60f-62cfdd82517f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:20.700930 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e658d-153b-4694-af20-59d9220d482d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:20.701855 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a37416-89d3-4bc1-b60f-62cfdd82517f admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:20.702540 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e658d-153b-4694-af20-59d9220d482d admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:20.720047 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 980/1978] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 20:03:20.725251 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a37416-89d3-4bc1-b60f-62cfdd82517f admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:20.736519 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a37416-89d3-4bc1-b60f-62cfdd82517f admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:20.737913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 999/1979] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:03:20 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:03:20.738981 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e658d-153b-4694-af20-59d9220d482d admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:20.804539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:20.811986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e658d-153b-4694-af20-59d9220d482d admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:20.817532 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a82828-020c-44d8-88cf-116335cd65a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:20.818303 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a82828-020c-44d8-88cf-116335cd65a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:20.818592 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a82828-020c-44d8-88cf-116335cd65a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:20.818865 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a82828-020c-44d8-88cf-116335cd65a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:20.819019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bc805-b21a-4497-a713-a42cc0bbcd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:20.819624 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a82828-020c-44d8-88cf-116335cd65a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:20.819933 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bc805-b21a-4497-a713-a42cc0bbcd47 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:20.820218 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bc805-b21a-4497-a713-a42cc0bbcd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:20.820490 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bc805-b21a-4497-a713-a42cc0bbcd47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:20.821221 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5bc805-b21a-4497-a713-a42cc0bbcd47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:20.821734 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a82828-020c-44d8-88cf-116335cd65a3 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:20.823215 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5bc805-b21a-4497-a713-a42cc0bbcd47 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:20.857733 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1000/1980] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:03:20 2022] POST /identity/v3/users => generated 287 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:20.859119 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a82828-020c-44d8-88cf-116335cd65a3 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:20.863741 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 981/1981] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:03:20 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:20.867875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:20.868204 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bc805-b21a-4497-a713-a42cc0bbcd47 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:20.873917 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a82828-020c-44d8-88cf-116335cd65a3 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:20.967866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d8579-5453-46f0-987e-110abf1611e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:20.968650 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d8579-5453-46f0-987e-110abf1611e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:20.969001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d8579-5453-46f0-987e-110abf1611e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:20.969632 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d8579-5453-46f0-987e-110abf1611e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:20.971028 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d8579-5453-46f0-987e-110abf1611e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:20.972864 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d8579-5453-46f0-987e-110abf1611e2 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:20.974491 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bc805-b21a-4497-a713-a42cc0bbcd47 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:20.980295 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b47d627-be8c-4dd9-8730-febbdac53e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:20.980958 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47d627-be8c-4dd9-8730-febbdac53e51 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:20.981240 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47d627-be8c-4dd9-8730-febbdac53e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:20.981655 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47d627-be8c-4dd9-8730-febbdac53e51 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:20.982329 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b47d627-be8c-4dd9-8730-febbdac53e51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:20.984060 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1001/1982] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:20.984334 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b47d627-be8c-4dd9-8730-febbdac53e51 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:20.988013 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d8579-5453-46f0-987e-110abf1611e2 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:20.998958 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d8579-5453-46f0-987e-110abf1611e2 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:21.006456 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e069315-213f-4205-9ff2-3c2dde0ca1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:21.006456 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e069315-213f-4205-9ff2-3c2dde0ca1d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:21.006929 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e069315-213f-4205-9ff2-3c2dde0ca1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:21.007325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e069315-213f-4205-9ff2-3c2dde0ca1d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:21.008094 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e069315-213f-4205-9ff2-3c2dde0ca1d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:21.009299 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 982/1983] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:03:20 2022] POST /identity/v3/users => generated 288 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:21.010239 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e069315-213f-4205-9ff2-3c2dde0ca1d3 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:21.013594 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b47d627-be8c-4dd9-8730-febbdac53e51 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:21.022015 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1002/1984] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:03:21.025389 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b47d627-be8c-4dd9-8730-febbdac53e51 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:21.026197 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e069315-213f-4205-9ff2-3c2dde0ca1d3 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:21.030876 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad4d971-706d-40fa-9cd0-603f8831a2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:21.031538 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4d971-706d-40fa-9cd0-603f8831a2b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:21.032420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4d971-706d-40fa-9cd0-603f8831a2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:21.032420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4d971-706d-40fa-9cd0-603f8831a2b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:21.032981 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad4d971-706d-40fa-9cd0-603f8831a2b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:21.035188 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad4d971-706d-40fa-9cd0-603f8831a2b1 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:21.036865 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e069315-213f-4205-9ff2-3c2dde0ca1d3 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:21.042332 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd03733c-8e8c-4dac-87a0-da1e1412330d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:21.043146 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03733c-8e8c-4dac-87a0-da1e1412330d admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:21.043431 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03733c-8e8c-4dac-87a0-da1e1412330d admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:21.043706 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03733c-8e8c-4dac-87a0-da1e1412330d admin admin] PATH_INFO: `/v3/projects/08cadf2cf9de44cf8f6cdd58dcb717c5/users/2a26ee339ae54dc0811ed24400fa9e79/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:21.047781 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 983/1985] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:21.051952 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad4d971-706d-40fa-9cd0-603f8831a2b1 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:21.066072 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad4d971-706d-40fa-9cd0-603f8831a2b1 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:21.066773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd03733c-8e8c-4dac-87a0-da1e1412330d admin admin] RBAC: Authorizing `identity:create_grant(project_id=08cadf2cf9de44cf8f6cdd58dcb717c5, user_id=2a26ee339ae54dc0811ed24400fa9e79, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:21.069582 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd03733c-8e8c-4dac-87a0-da1e1412330d admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:21.071423 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82bb66d-0d29-41a4-9f50-cfc460954089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:21.072041 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bb66d-0d29-41a4-9f50-cfc460954089 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:21.072326 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bb66d-0d29-41a4-9f50-cfc460954089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:21.072768 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bb66d-0d29-41a4-9f50-cfc460954089 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:21.073589 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82bb66d-0d29-41a4-9f50-cfc460954089 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:21.075956 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82bb66d-0d29-41a4-9f50-cfc460954089 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:21.085257 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1003/1986] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:21 2022] PUT /identity/v3/projects/08cadf2cf9de44cf8f6cdd58dcb717c5/users/2a26ee339ae54dc0811ed24400fa9e79/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:03:21.092282 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 984/1987] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:21.092423 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36cb05-460b-4f80-9024-feee3cb7a318 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:21.092423 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36cb05-460b-4f80-9024-feee3cb7a318 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:21.092423 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36cb05-460b-4f80-9024-feee3cb7a318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:21.093309 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82bb66d-0d29-41a4-9f50-cfc460954089 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:21.104250 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82bb66d-0d29-41a4-9f50-cfc460954089 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:21.115003 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-5b36cb05-460b-4f80-9024-feee3cb7a318 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:21.116468 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5b36cb05-460b-4f80-9024-feee3cb7a318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:21.170001 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed79ab0-b045-4df8-8b94-5bdb27387cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:21.170714 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed79ab0-b045-4df8-8b94-5bdb27387cec admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:21.170995 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed79ab0-b045-4df8-8b94-5bdb27387cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:21.171274 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed79ab0-b045-4df8-8b94-5bdb27387cec admin admin] PATH_INFO: `/v3/projects/37778508d3ec49cab34cbbc5b222ba2b/users/a8253453c662476dbcb9c95f36bc3ad7/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:21.182237 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1004/1988] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:21.186484 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed79ab0-b045-4df8-8b94-5bdb27387cec admin admin] RBAC: Authorizing `identity:create_grant(project_id=37778508d3ec49cab34cbbc5b222ba2b, user_id=a8253453c662476dbcb9c95f36bc3ad7, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:21.189401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed79ab0-b045-4df8-8b94-5bdb27387cec admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:21.194672 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b36cb05-460b-4f80-9024-feee3cb7a318 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:21.196192 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5b36cb05-460b-4f80-9024-feee3cb7a318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:21.206688 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 985/1989] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:21 2022] PUT /identity/v3/projects/37778508d3ec49cab34cbbc5b222ba2b/users/a8253453c662476dbcb9c95f36bc3ad7/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:21.217750 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba431bc4-675b-4d5b-ab88-6f085a06668c None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:21.218039 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba431bc4-675b-4d5b-ab88-6f085a06668c None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:21.218313 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba431bc4-675b-4d5b-ab88-6f085a06668c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:21.249615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-ba431bc4-675b-4d5b-ab88-6f085a06668c None None] MFA Rules not processed for user `a8253453c662476dbcb9c95f36bc3ad7`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:21.250820 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-ba431bc4-675b-4d5b-ab88-6f085a06668c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:21.260895 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b36cb05-460b-4f80-9024-feee3cb7a318 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:21.266457 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:21.280032 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:21.322287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16adbe4d-7d2c-474f-802d-5471a00d14f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:21.323167 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16adbe4d-7d2c-474f-802d-5471a00d14f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:21.323456 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16adbe4d-7d2c-474f-802d-5471a00d14f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:21.323730 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16adbe4d-7d2c-474f-802d-5471a00d14f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:21.336636 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 986/1990] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:03:21.344205 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba431bc4-675b-4d5b-ab88-6f085a06668c None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:21.345616 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-ba431bc4-675b-4d5b-ab88-6f085a06668c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:21.411745 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16adbe4d-7d2c-474f-802d-5471a00d14f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:21.413130 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16adbe4d-7d2c-474f-802d-5471a00d14f0 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:21.423323 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba431bc4-675b-4d5b-ab88-6f085a06668c None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:21.425106 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1005/1991] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:03:21.428620 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d220d6-e5ad-433b-b230-a12dee865594 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:21.429244 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d220d6-e5ad-433b-b230-a12dee865594 service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:21.429526 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d220d6-e5ad-433b-b230-a12dee865594 service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:21.429804 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d220d6-e5ad-433b-b230-a12dee865594 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:21.439216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d220d6-e5ad-433b-b230-a12dee865594 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:21.445532 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d220d6-e5ad-433b-b230-a12dee865594 service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:21.458789 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 987/1992] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:03:22.108223 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d49e0-06ac-427a-b41a-2e29a4ee3522 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:22.108563 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d49e0-06ac-427a-b41a-2e29a4ee3522 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:22.108883 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d49e0-06ac-427a-b41a-2e29a4ee3522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:22.146393 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-596d49e0-06ac-427a-b41a-2e29a4ee3522 None None] MFA Rules not processed for user `2a26ee339ae54dc0811ed24400fa9e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:22.151907 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-596d49e0-06ac-427a-b41a-2e29a4ee3522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:22.230087 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1006/1993] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:03:22.234189 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d220d6-e5ad-433b-b230-a12dee865594 service cinder] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:22.247676 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d220d6-e5ad-433b-b230-a12dee865594 service cinder] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:22.254775 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b9a1b-fd90-43e0-b559-45cacf7cf00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:22.254775 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b9a1b-fd90-43e0-b559-45cacf7cf00a admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:22.255734 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b9a1b-fd90-43e0-b559-45cacf7cf00a admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:22.256021 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b9a1b-fd90-43e0-b559-45cacf7cf00a admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:22.256743 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b9a1b-fd90-43e0-b559-45cacf7cf00a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:22.258641 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b9a1b-fd90-43e0-b559-45cacf7cf00a admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:22.298897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 988/1994] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:03:22 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:22.303594 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d49e0-06ac-427a-b41a-2e29a4ee3522 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:22.314499 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d49e0-06ac-427a-b41a-2e29a4ee3522 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:22.370875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc0a53-a699-405f-9f12-a3f984e73114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:22.371490 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc0a53-a699-405f-9f12-a3f984e73114 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:22.371773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc0a53-a699-405f-9f12-a3f984e73114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:22.372049 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc0a53-a699-405f-9f12-a3f984e73114 admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:22.372760 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bc0a53-a699-405f-9f12-a3f984e73114 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:22.374683 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bc0a53-a699-405f-9f12-a3f984e73114 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:22.382866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:22.398600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1007/1995] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:03:22 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:03:22.402452 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b9a1b-fd90-43e0-b559-45cacf7cf00a admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:22.413523 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b9a1b-fd90-43e0-b559-45cacf7cf00a admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:22.418498 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2649ec9-4b1a-42d7-b2bf-e52a308e20e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:22.419121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2649ec9-4b1a-42d7-b2bf-e52a308e20e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:22.419453 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2649ec9-4b1a-42d7-b2bf-e52a308e20e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:22.419733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2649ec9-4b1a-42d7-b2bf-e52a308e20e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:22.420427 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2649ec9-4b1a-42d7-b2bf-e52a308e20e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:22.422080 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2649ec9-4b1a-42d7-b2bf-e52a308e20e6 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:22.433108 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 989/1996] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:03:22.437334 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc0a53-a699-405f-9f12-a3f984e73114 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:22.448392 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc0a53-a699-405f-9f12-a3f984e73114 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:22.452912 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0ea5a-cf5d-4ef8-8a04-594cd8783700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:22.453593 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0ea5a-cf5d-4ef8-8a04-594cd8783700 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:22.453871 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0ea5a-cf5d-4ef8-8a04-594cd8783700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:22.454141 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0ea5a-cf5d-4ef8-8a04-594cd8783700 admin admin] PATH_INFO: `/v3/projects/cb0905767c48451494dddf9c3008edf5/users/bc6a15df3d39474f94addf30c4ed9d63/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:22.471575 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf0ea5a-cf5d-4ef8-8a04-594cd8783700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb0905767c48451494dddf9c3008edf5, user_id=bc6a15df3d39474f94addf30c4ed9d63, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:22.475098 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf0ea5a-cf5d-4ef8-8a04-594cd8783700 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:22.488866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1008/1997] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:22 2022] PUT /identity/v3/projects/cb0905767c48451494dddf9c3008edf5/users/bc6a15df3d39474f94addf30c4ed9d63/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:22.493198 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2649ec9-4b1a-42d7-b2bf-e52a308e20e6 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:22.503580 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2649ec9-4b1a-42d7-b2bf-e52a308e20e6 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:22.592004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134679fc-2da5-4cee-b45f-e051ca7e3399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:22.592660 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134679fc-2da5-4cee-b45f-e051ca7e3399 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:22.593172 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134679fc-2da5-4cee-b45f-e051ca7e3399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:22.593458 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134679fc-2da5-4cee-b45f-e051ca7e3399 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:22.594224 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134679fc-2da5-4cee-b45f-e051ca7e3399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:22.596478 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134679fc-2da5-4cee-b45f-e051ca7e3399 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:22.608616 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 990/1998] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:22.613816 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0ea5a-cf5d-4ef8-8a04-594cd8783700 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:22.624779 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0ea5a-cf5d-4ef8-8a04-594cd8783700 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:22.630206 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf8bd8-7c1a-40a5-a4d4-fcbb88b4376d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:22.630881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf8bd8-7c1a-40a5-a4d4-fcbb88b4376d admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:22.631161 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf8bd8-7c1a-40a5-a4d4-fcbb88b4376d admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:22.631440 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf8bd8-7c1a-40a5-a4d4-fcbb88b4376d admin admin] PATH_INFO: `/v3/projects/cb0905767c48451494dddf9c3008edf5/users/bc6a15df3d39474f94addf30c4ed9d63/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:22.633441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edf8bd8-7c1a-40a5-a4d4-fcbb88b4376d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb0905767c48451494dddf9c3008edf5, user_id=bc6a15df3d39474f94addf30c4ed9d63, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:22.636351 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edf8bd8-7c1a-40a5-a4d4-fcbb88b4376d admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:22.650351 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1009/1999] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:22 2022] PUT /identity/v3/projects/cb0905767c48451494dddf9c3008edf5/users/bc6a15df3d39474f94addf30c4ed9d63/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:23.408341 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17025b-dcef-408e-af90-6a3a6a91c2e5 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:23.409209 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17025b-dcef-408e-af90-6a3a6a91c2e5 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:23.409209 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17025b-dcef-408e-af90-6a3a6a91c2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:23.457748 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-5d17025b-dcef-408e-af90-6a3a6a91c2e5 None None] MFA Rules not processed for user `bc6a15df3d39474f94addf30c4ed9d63`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:23.458658 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-5d17025b-dcef-408e-af90-6a3a6a91c2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:23.470999 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:23.552426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 991/2000] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:23 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:23.564225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf8bd8-7c1a-40a5-a4d4-fcbb88b4376d admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:23.640350 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf8bd8-7c1a-40a5-a4d4-fcbb88b4376d admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:23.645043 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd53f5-9a79-4d98-a2d7-a24651e895cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:23.646091 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd53f5-9a79-4d98-a2d7-a24651e895cd service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:23.646382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd53f5-9a79-4d98-a2d7-a24651e895cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:23.646678 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd53f5-9a79-4d98-a2d7-a24651e895cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:23.747947 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd53f5-9a79-4d98-a2d7-a24651e895cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:23.749893 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd53f5-9a79-4d98-a2d7-a24651e895cd service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:23.768681 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1010/2001] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:03:24.325879 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25029b-a661-4829-b363-2b751fcfa1cc None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:24.326245 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25029b-a661-4829-b363-2b751fcfa1cc None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:24.326528 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25029b-a661-4829-b363-2b751fcfa1cc None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:24.327901 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 992/2002] 10.222.0.39 () {54 vars in 921 bytes} [Fri Aug 5 20:03:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 20:03:24.350660 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d06991-097c-468c-857a-47dc2e163f9b None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:24.350951 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d06991-097c-468c-857a-47dc2e163f9b None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:24.351237 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d06991-097c-468c-857a-47dc2e163f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:24.380607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-44d06991-097c-468c-857a-47dc2e163f9b None None] MFA Rules not processed for user `95e9f085e4f04d118373b8abff27543a`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:24.381967 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-44d06991-097c-468c-857a-47dc2e163f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:24.423252 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25029b-a661-4829-b363-2b751fcfa1cc None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:24.437585 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25029b-a661-4829-b363-2b751fcfa1cc None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:24.442177 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf47a8-c4f4-4489-9df2-647d9ef493db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:24.442797 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf47a8-c4f4-4489-9df2-647d9ef493db service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:24.443301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf47a8-c4f4-4489-9df2-647d9ef493db service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:24.443576 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf47a8-c4f4-4489-9df2-647d9ef493db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:24.445730 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-bcdf47a8-c4f4-4489-9df2-647d9ef493db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:24.474140 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:24.492373 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1011/2003] 10.222.0.39 () {58 vars in 1018 bytes} [Fri Aug 5 20:03:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:03:24.505868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d06991-097c-468c-857a-47dc2e163f9b None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:24.507486 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-44d06991-097c-468c-857a-47dc2e163f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:24.519636 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf47a8-c4f4-4489-9df2-647d9ef493db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:24.522049 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf47a8-c4f4-4489-9df2-647d9ef493db service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:24.525584 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d06991-097c-468c-857a-47dc2e163f9b None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:24.532028 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce30fc1-6539-48d0-859d-4da2a218ed9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:24.532717 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce30fc1-6539-48d0-859d-4da2a218ed9b service nova] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:24.533496 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce30fc1-6539-48d0-859d-4da2a218ed9b service nova] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:24.533774 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce30fc1-6539-48d0-859d-4da2a218ed9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:24.543788 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce30fc1-6539-48d0-859d-4da2a218ed9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:24.545448 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce30fc1-6539-48d0-859d-4da2a218ed9b service nova] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:24.553504 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 993/2004] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:03:24.565496 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1012/2005] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:03:25.129197 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf47a8-c4f4-4489-9df2-647d9ef493db service cinder] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:25.130687 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-bcdf47a8-c4f4-4489-9df2-647d9ef493db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:25.205926 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf47a8-c4f4-4489-9df2-647d9ef493db service cinder] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:25.210659 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa865a-d615-426f-838a-3ba86fba8de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:25.211294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa865a-d615-426f-838a-3ba86fba8de4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:25.211569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa865a-d615-426f-838a-3ba86fba8de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:25.211841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa865a-d615-426f-838a-3ba86fba8de4 admin admin] PATH_INFO: `/v3/users/9799d7c72ad540eb9dafacdeefb595a3` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:25.213018 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aa865a-d615-426f-838a-3ba86fba8de4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9799d7c72ad540eb9dafacdeefb595a3)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:25.214919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aa865a-d615-426f-838a-3ba86fba8de4 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:25.281251 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-27aa865a-d615-426f-838a-3ba86fba8de4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9799d7c72ad540eb9dafacdeefb595a3'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:25.283952 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:25.292905 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-27aa865a-d615-426f-838a-3ba86fba8de4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9799d7c72ad540eb9dafacdeefb595a3'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:25.307121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-27aa865a-d615-426f-838a-3ba86fba8de4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9799d7c72ad540eb9dafacdeefb595a3'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:25.307884 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-27aa865a-d615-426f-838a-3ba86fba8de4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9799d7c72ad540eb9dafacdeefb595a3'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:25.315785 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-27aa865a-d615-426f-838a-3ba86fba8de4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9799d7c72ad540eb9dafacdeefb595a3'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:25.324843 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 994/2006] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:03:25 2022] DELETE /identity/v3/users/9799d7c72ad540eb9dafacdeefb595a3 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:25.329129 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71bb771-e97d-4aec-b994-892c09a38f83 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:25.329411 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71bb771-e97d-4aec-b994-892c09a38f83 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:25.329681 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71bb771-e97d-4aec-b994-892c09a38f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:25.350497 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-a71bb771-e97d-4aec-b994-892c09a38f83 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:25.351951 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-a71bb771-e97d-4aec-b994-892c09a38f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:25.416406 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1013/2007] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:25.426664 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa865a-d615-426f-838a-3ba86fba8de4 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:25.428098 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-27aa865a-d615-426f-838a-3ba86fba8de4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:25.485680 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa865a-d615-426f-838a-3ba86fba8de4 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:25.489725 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7e976-97be-4c48-8e48-6e273c00aeb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:25.490340 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7e976-97be-4c48-8e48-6e273c00aeb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:25.490623 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7e976-97be-4c48-8e48-6e273c00aeb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:25.490894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7e976-97be-4c48-8e48-6e273c00aeb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:25.499723 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7e976-97be-4c48-8e48-6e273c00aeb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:25.501228 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7e976-97be-4c48-8e48-6e273c00aeb9 service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:25.511235 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 995/2008] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:03:25.788746 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71bb771-e97d-4aec-b994-892c09a38f83 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:25.790716 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-a71bb771-e97d-4aec-b994-892c09a38f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:25.803759 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71bb771-e97d-4aec-b994-892c09a38f83 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:25.808315 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68f6a8-9a7e-49ca-a9cf-f21f941f4467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:25.808982 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68f6a8-9a7e-49ca-a9cf-f21f941f4467 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:25.809262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68f6a8-9a7e-49ca-a9cf-f21f941f4467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:25.809531 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68f6a8-9a7e-49ca-a9cf-f21f941f4467 admin admin] PATH_INFO: `/v3/projects/6e24b112a9bb4e529b1fe3c8be573ed4` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:25.810671 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68f6a8-9a7e-49ca-a9cf-f21f941f4467 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e24b112a9bb4e529b1fe3c8be573ed4)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:25.812566 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68f6a8-9a7e-49ca-a9cf-f21f941f4467 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:25.858907 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-fa68f6a8-9a7e-49ca-a9cf-f21f941f4467 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e24b112a9bb4e529b1fe3c8be573ed4', 'request_id': 'req-fa68f6a8-9a7e-49ca-a9cf-f21f941f4467'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:25.865626 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-fa68f6a8-9a7e-49ca-a9cf-f21f941f4467 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e24b112a9bb4e529b1fe3c8be573ed4', 'request_id': 'req-fa68f6a8-9a7e-49ca-a9cf-f21f941f4467'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:25.872768 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-fa68f6a8-9a7e-49ca-a9cf-f21f941f4467 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e24b112a9bb4e529b1fe3c8be573ed4', 'request_id': 'req-fa68f6a8-9a7e-49ca-a9cf-f21f941f4467'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:25.884089 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-fa68f6a8-9a7e-49ca-a9cf-f21f941f4467 admin admin] The token cache is being invalidate because project 6e24b112a9bb4e529b1fe3c8be573ed4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:03:25.884381 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-fa68f6a8-9a7e-49ca-a9cf-f21f941f4467 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:25.885048 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-fa68f6a8-9a7e-49ca-a9cf-f21f941f4467 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:25.887089 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1014/2009] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:03:25 2022] DELETE /identity/v3/projects/6e24b112a9bb4e529b1fe3c8be573ed4 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:03:25.890980 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7e976-97be-4c48-8e48-6e273c00aeb9 service neutron] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:25.901076 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7e976-97be-4c48-8e48-6e273c00aeb9 service neutron] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:25.960889 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad9fe7d-f9b8-4f4a-b698-09805fc52fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:25.961535 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9fe7d-f9b8-4f4a-b698-09805fc52fb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:25.961815 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9fe7d-f9b8-4f4a-b698-09805fc52fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:25.962086 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9fe7d-f9b8-4f4a-b698-09805fc52fb5 admin admin] PATH_INFO: `/v3/users/b3d2f631881e46b5bffaae677c1dad90` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:25.963246 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad9fe7d-f9b8-4f4a-b698-09805fc52fb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3d2f631881e46b5bffaae677c1dad90)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:25.965230 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad9fe7d-f9b8-4f4a-b698-09805fc52fb5 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:26.017974 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-bad9fe7d-f9b8-4f4a-b698-09805fc52fb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3d2f631881e46b5bffaae677c1dad90'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:26.026421 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-bad9fe7d-f9b8-4f4a-b698-09805fc52fb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3d2f631881e46b5bffaae677c1dad90'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:26.044547 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:26.047480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-bad9fe7d-f9b8-4f4a-b698-09805fc52fb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3d2f631881e46b5bffaae677c1dad90'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:26.048216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-bad9fe7d-f9b8-4f4a-b698-09805fc52fb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3d2f631881e46b5bffaae677c1dad90'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:26.055425 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-bad9fe7d-f9b8-4f4a-b698-09805fc52fb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3d2f631881e46b5bffaae677c1dad90'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:26.063605 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 996/2010] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:03:25 2022] DELETE /identity/v3/users/b3d2f631881e46b5bffaae677c1dad90 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:26.372955 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68f6a8-9a7e-49ca-a9cf-f21f941f4467 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:26.372955 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-fa68f6a8-9a7e-49ca-a9cf-f21f941f4467 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:26.412921 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68f6a8-9a7e-49ca-a9cf-f21f941f4467 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:26.498912 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb581d0-2ff5-4bf5-8554-62b20791dd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:26.499179 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb581d0-2ff5-4bf5-8554-62b20791dd72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:26.499473 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb581d0-2ff5-4bf5-8554-62b20791dd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:26.500114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb581d0-2ff5-4bf5-8554-62b20791dd72 admin admin] PATH_INFO: `/v3/projects/3c989e137c3b4803929fdb0433419c60` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:26.500978 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb581d0-2ff5-4bf5-8554-62b20791dd72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c989e137c3b4803929fdb0433419c60)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:26.503528 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb581d0-2ff5-4bf5-8554-62b20791dd72 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:26.514841 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:26.563170 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5cb581d0-2ff5-4bf5-8554-62b20791dd72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c989e137c3b4803929fdb0433419c60', 'request_id': 'req-5cb581d0-2ff5-4bf5-8554-62b20791dd72'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:26.570078 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5cb581d0-2ff5-4bf5-8554-62b20791dd72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c989e137c3b4803929fdb0433419c60', 'request_id': 'req-5cb581d0-2ff5-4bf5-8554-62b20791dd72'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:26.577856 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5cb581d0-2ff5-4bf5-8554-62b20791dd72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c989e137c3b4803929fdb0433419c60', 'request_id': 'req-5cb581d0-2ff5-4bf5-8554-62b20791dd72'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:26.589112 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5cb581d0-2ff5-4bf5-8554-62b20791dd72 admin admin] The token cache is being invalidate because project 3c989e137c3b4803929fdb0433419c60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:03:26.589470 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5cb581d0-2ff5-4bf5-8554-62b20791dd72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:26.590179 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5cb581d0-2ff5-4bf5-8554-62b20791dd72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:26.591766 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1015/2011] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:03:26 2022] DELETE /identity/v3/projects/3c989e137c3b4803929fdb0433419c60 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:26.596868 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc7aaf-48af-4cdb-bde9-c19e1dfd35fd None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:26.597371 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc7aaf-48af-4cdb-bde9-c19e1dfd35fd None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:26.597700 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc7aaf-48af-4cdb-bde9-c19e1dfd35fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:26.620310 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-fcdc7aaf-48af-4cdb-bde9-c19e1dfd35fd None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:26.621618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-fcdc7aaf-48af-4cdb-bde9-c19e1dfd35fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:26.680822 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb581d0-2ff5-4bf5-8554-62b20791dd72 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:26.682289 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5cb581d0-2ff5-4bf5-8554-62b20791dd72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:26.692817 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 997/2012] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:26.700524 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9acb6a3-7c85-48da-99b5-e9b439770ea0 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:26.700849 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9acb6a3-7c85-48da-99b5-e9b439770ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:26.701121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9acb6a3-7c85-48da-99b5-e9b439770ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:26.719699 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-f9acb6a3-7c85-48da-99b5-e9b439770ea0 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:26.721533 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-f9acb6a3-7c85-48da-99b5-e9b439770ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:26.729416 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 998/2013] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:03:26.733484 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9acb6a3-7c85-48da-99b5-e9b439770ea0 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:26.742557 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9acb6a3-7c85-48da-99b5-e9b439770ea0 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:26.747098 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865bc93-7fbe-4f59-bab2-6b368b38c293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:26.747783 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865bc93-7fbe-4f59-bab2-6b368b38c293 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:26.748086 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865bc93-7fbe-4f59-bab2-6b368b38c293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:26.748361 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865bc93-7fbe-4f59-bab2-6b368b38c293 admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:26.749071 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7865bc93-7fbe-4f59-bab2-6b368b38c293 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:03:26.749403 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7865bc93-7fbe-4f59-bab2-6b368b38c293 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:26.750994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7865bc93-7fbe-4f59-bab2-6b368b38c293 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:26.751515 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb581d0-2ff5-4bf5-8554-62b20791dd72 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:26.757379 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e25ab0-80c9-4743-8d37-47d807884d3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:26.758024 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e25ab0-80c9-4743-8d37-47d807884d3a service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:26.758304 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e25ab0-80c9-4743-8d37-47d807884d3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:26.758580 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e25ab0-80c9-4743-8d37-47d807884d3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:26.760348 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e0e25ab0-80c9-4743-8d37-47d807884d3a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:26.763555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 999/2014] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:03:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 20:03:26.767968 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865bc93-7fbe-4f59-bab2-6b368b38c293 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:26.777621 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865bc93-7fbe-4f59-bab2-6b368b38c293 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:26.782353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ddb00-a34d-43ac-800c-57df30e8a052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:26.782948 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ddb00-a34d-43ac-800c-57df30e8a052 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:26.783753 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ddb00-a34d-43ac-800c-57df30e8a052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:26.784017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ddb00-a34d-43ac-800c-57df30e8a052 admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:26.784699 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ddb00-a34d-43ac-800c-57df30e8a052 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:26.786569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ddb00-a34d-43ac-800c-57df30e8a052 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:26.813687 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1000/2015] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:03:26 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:03:26.818431 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ddb00-a34d-43ac-800c-57df30e8a052 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:26.826180 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e25ab0-80c9-4743-8d37-47d807884d3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:26.827626 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e25ab0-80c9-4743-8d37-47d807884d3a service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:26.837671 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1016/2016] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:26 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:03:26.881249 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ddb00-a34d-43ac-800c-57df30e8a052 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:26.885996 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c1439-60bb-4605-9673-bfe53be83e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:26.886603 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c1439-60bb-4605-9673-bfe53be83e6d admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:26.887022 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c1439-60bb-4605-9673-bfe53be83e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:26.887299 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c1439-60bb-4605-9673-bfe53be83e6d admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:26.887986 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c1439-60bb-4605-9673-bfe53be83e6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:26.889918 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c1439-60bb-4605-9673-bfe53be83e6d admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:26.912865 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1001/2017] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:03:26 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:03:26.917271 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e25ab0-80c9-4743-8d37-47d807884d3a service cinder] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:26.929614 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e25ab0-80c9-4743-8d37-47d807884d3a service cinder] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:26.934315 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367af907-1973-40bd-98a2-1bd5f870299a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:26.934933 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367af907-1973-40bd-98a2-1bd5f870299a admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:26.935220 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367af907-1973-40bd-98a2-1bd5f870299a admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:26.935498 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367af907-1973-40bd-98a2-1bd5f870299a admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:26.936199 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367af907-1973-40bd-98a2-1bd5f870299a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:26.937880 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367af907-1973-40bd-98a2-1bd5f870299a admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:26.950200 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1017/2018] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:26.953815 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c1439-60bb-4605-9673-bfe53be83e6d admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:26.965027 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c1439-60bb-4605-9673-bfe53be83e6d admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:26.969710 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c130e-70fd-47c2-80d3-3e9e67fce1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:26.970318 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c130e-70fd-47c2-80d3-3e9e67fce1d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:26.970604 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c130e-70fd-47c2-80d3-3e9e67fce1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:26.971417 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c130e-70fd-47c2-80d3-3e9e67fce1d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:26.972106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c130e-70fd-47c2-80d3-3e9e67fce1d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:26.974755 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c130e-70fd-47c2-80d3-3e9e67fce1d9 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:26.985467 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1002/2019] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:03:26.989759 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367af907-1973-40bd-98a2-1bd5f870299a admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:26.999732 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367af907-1973-40bd-98a2-1bd5f870299a admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:27.005132 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee77a2cf-32ee-4dfb-9a2d-f52a41824fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:27.005792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77a2cf-32ee-4dfb-9a2d-f52a41824fbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:27.006076 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77a2cf-32ee-4dfb-9a2d-f52a41824fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:27.006356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77a2cf-32ee-4dfb-9a2d-f52a41824fbf admin admin] PATH_INFO: `/v3/projects/fafe2f672b7c4979bbc76dc9445bc628/users/c1c66c943ea14a3c852843c598559144/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:27.021869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee77a2cf-32ee-4dfb-9a2d-f52a41824fbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=fafe2f672b7c4979bbc76dc9445bc628, user_id=c1c66c943ea14a3c852843c598559144, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:27.024519 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee77a2cf-32ee-4dfb-9a2d-f52a41824fbf admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:27.037378 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1018/2020] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:26 2022] PUT /identity/v3/projects/fafe2f672b7c4979bbc76dc9445bc628/users/c1c66c943ea14a3c852843c598559144/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:27.047480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b4b6c-3dae-4b96-8ce6-df7de9687a83 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:27.047767 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b4b6c-3dae-4b96-8ce6-df7de9687a83 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:27.048138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b4b6c-3dae-4b96-8ce6-df7de9687a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:27.077540 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-b58b4b6c-3dae-4b96-8ce6-df7de9687a83 None None] MFA Rules not processed for user `c1c66c943ea14a3c852843c598559144`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:27.078761 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-b58b4b6c-3dae-4b96-8ce6-df7de9687a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:27.092260 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:27.150189 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1003/2021] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:27.161461 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee77a2cf-32ee-4dfb-9a2d-f52a41824fbf admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:27.163178 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-ee77a2cf-32ee-4dfb-9a2d-f52a41824fbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:27.181831 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:27.249836 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee77a2cf-32ee-4dfb-9a2d-f52a41824fbf admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:27.253961 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56726ab2-c238-45d8-9b57-e5390f8efd7d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe69b398c1f4e18b625a63ae40b01e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:27.254627 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56726ab2-c238-45d8-9b57-e5390f8efd7d service glance] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:27.254909 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56726ab2-c238-45d8-9b57-e5390f8efd7d service glance] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:27.255185 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56726ab2-c238-45d8-9b57-e5390f8efd7d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:27.264528 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56726ab2-c238-45d8-9b57-e5390f8efd7d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:27.265981 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56726ab2-c238-45d8-9b57-e5390f8efd7d service glance] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:27.275865 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1019/2022] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:03:30.613017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b4b6c-3dae-4b96-8ce6-df7de9687a83 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:30.614428 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-b58b4b6c-3dae-4b96-8ce6-df7de9687a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:30.684253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b4b6c-3dae-4b96-8ce6-df7de9687a83 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:30.690964 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de5929-6aef-4169-9b63-305c73dbdd71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:30.691844 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de5929-6aef-4169-9b63-305c73dbdd71 service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:30.692257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de5929-6aef-4169-9b63-305c73dbdd71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:30.692794 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de5929-6aef-4169-9b63-305c73dbdd71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:30.700227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-07de5929-6aef-4169-9b63-305c73dbdd71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:30.803632 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de5929-6aef-4169-9b63-305c73dbdd71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:30.805293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de5929-6aef-4169-9b63-305c73dbdd71 service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:30.816159 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1004/2023] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:30 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:03:37.271759 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=29 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:39.507292 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56726ab2-c238-45d8-9b57-e5390f8efd7d service glance] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:39.508827 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-56726ab2-c238-45d8-9b57-e5390f8efd7d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:39.579312 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56726ab2-c238-45d8-9b57-e5390f8efd7d service glance] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:39.585117 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d458d5db-8697-418d-bb13-eab60ef097c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:39.585790 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458d5db-8697-418d-bb13-eab60ef097c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:39.586074 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458d5db-8697-418d-bb13-eab60ef097c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:39.586345 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458d5db-8697-418d-bb13-eab60ef097c4 admin admin] PATH_INFO: `/v3/users/345aa188de3b49f5b5205983764e152e` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:39.587694 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d458d5db-8697-418d-bb13-eab60ef097c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=345aa188de3b49f5b5205983764e152e)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:39.589489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d458d5db-8697-418d-bb13-eab60ef097c4 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:39.636940 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-d458d5db-8697-418d-bb13-eab60ef097c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '345aa188de3b49f5b5205983764e152e'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:39.645338 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-d458d5db-8697-418d-bb13-eab60ef097c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '345aa188de3b49f5b5205983764e152e'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:39.652463 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-d458d5db-8697-418d-bb13-eab60ef097c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '345aa188de3b49f5b5205983764e152e'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:39.665035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-d458d5db-8697-418d-bb13-eab60ef097c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '345aa188de3b49f5b5205983764e152e'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:39.665670 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-d458d5db-8697-418d-bb13-eab60ef097c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '345aa188de3b49f5b5205983764e152e'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:39.671547 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1020/2024] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:03:39 2022] DELETE /identity/v3/users/345aa188de3b49f5b5205983764e152e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:39.676478 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e0c99-f229-48b4-b7b7-b61ade3ab607 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:39.676995 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e0c99-f229-48b4-b7b7-b61ade3ab607 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:39.677102 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e0c99-f229-48b4-b7b7-b61ade3ab607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:39.697557 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-757e0c99-f229-48b4-b7b7-b61ade3ab607 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:39.698759 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-757e0c99-f229-48b4-b7b7-b61ade3ab607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:39.761747 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1005/2025] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:39.774100 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d458d5db-8697-418d-bb13-eab60ef097c4 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:39.779346 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-d458d5db-8697-418d-bb13-eab60ef097c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:39.865284 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d458d5db-8697-418d-bb13-eab60ef097c4 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:39.870980 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c18e6-0748-41d9-b457-d463b557238f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:39.871649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c18e6-0748-41d9-b457-d463b557238f service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:39.871997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c18e6-0748-41d9-b457-d463b557238f service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:39.872329 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c18e6-0748-41d9-b457-d463b557238f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:39.891011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9c18e6-0748-41d9-b457-d463b557238f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:39.892725 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9c18e6-0748-41d9-b457-d463b557238f service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:39.907877 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1021/2026] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:03:40.211333 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e0c99-f229-48b4-b7b7-b61ade3ab607 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:40.213390 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-757e0c99-f229-48b4-b7b7-b61ade3ab607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:40.241535 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:40.247342 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e0c99-f229-48b4-b7b7-b61ade3ab607 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:40.253295 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a4dece-a904-4617-9955-eee14777b1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:40.253941 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a4dece-a904-4617-9955-eee14777b1ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:40.254681 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a4dece-a904-4617-9955-eee14777b1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:40.254681 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a4dece-a904-4617-9955-eee14777b1ba admin admin] PATH_INFO: `/v3/projects/44794c3479904067b96e6519a8118f6c` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:40.255926 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a4dece-a904-4617-9955-eee14777b1ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=44794c3479904067b96e6519a8118f6c)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:40.258247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a4dece-a904-4617-9955-eee14777b1ba admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:40.334488 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-b5a4dece-a904-4617-9955-eee14777b1ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44794c3479904067b96e6519a8118f6c', 'request_id': 'req-b5a4dece-a904-4617-9955-eee14777b1ba'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:40.348390 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-b5a4dece-a904-4617-9955-eee14777b1ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44794c3479904067b96e6519a8118f6c', 'request_id': 'req-b5a4dece-a904-4617-9955-eee14777b1ba'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:40.364773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-b5a4dece-a904-4617-9955-eee14777b1ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44794c3479904067b96e6519a8118f6c', 'request_id': 'req-b5a4dece-a904-4617-9955-eee14777b1ba'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:40.376770 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-b5a4dece-a904-4617-9955-eee14777b1ba admin admin] The token cache is being invalidate because project 44794c3479904067b96e6519a8118f6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:03:40.377193 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-b5a4dece-a904-4617-9955-eee14777b1ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:40.378097 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-b5a4dece-a904-4617-9955-eee14777b1ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:40.383817 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1006/2027] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:03:40 2022] DELETE /identity/v3/projects/44794c3479904067b96e6519a8118f6c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:03:40.390812 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c18e6-0748-41d9-b457-d463b557238f service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:40.404100 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c18e6-0748-41d9-b457-d463b557238f service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:40.460751 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:40.531644 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319aa7b5-f65b-4b7f-92ae-a77587337086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:40.533864 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319aa7b5-f65b-4b7f-92ae-a77587337086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:40.534317 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319aa7b5-f65b-4b7f-92ae-a77587337086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:40.534731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319aa7b5-f65b-4b7f-92ae-a77587337086 admin admin] PATH_INFO: `/v3/users/caa1532321c146b39e9b99e16c62772d` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:40.537013 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319aa7b5-f65b-4b7f-92ae-a77587337086 admin admin] RBAC: Authorizing `identity:delete_user(user_id=caa1532321c146b39e9b99e16c62772d)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:40.542036 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319aa7b5-f65b-4b7f-92ae-a77587337086 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:40.621416 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-319aa7b5-f65b-4b7f-92ae-a77587337086 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caa1532321c146b39e9b99e16c62772d'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:40.629712 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-319aa7b5-f65b-4b7f-92ae-a77587337086 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caa1532321c146b39e9b99e16c62772d'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:40.638064 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-319aa7b5-f65b-4b7f-92ae-a77587337086 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caa1532321c146b39e9b99e16c62772d'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:40.650779 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-319aa7b5-f65b-4b7f-92ae-a77587337086 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caa1532321c146b39e9b99e16c62772d'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:40.651473 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-319aa7b5-f65b-4b7f-92ae-a77587337086 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caa1532321c146b39e9b99e16c62772d'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:40.658255 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1022/2028] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:03:40 2022] DELETE /identity/v3/users/caa1532321c146b39e9b99e16c62772d => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:40.981548 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a4dece-a904-4617-9955-eee14777b1ba admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:40.982960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-b5a4dece-a904-4617-9955-eee14777b1ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:40.996625 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a4dece-a904-4617-9955-eee14777b1ba admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:41.077244 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c0ea2c-ae4f-490f-bd80-868fb8324f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:41.078140 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0ea2c-ae4f-490f-bd80-868fb8324f82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:41.078441 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0ea2c-ae4f-490f-bd80-868fb8324f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:41.078703 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0ea2c-ae4f-490f-bd80-868fb8324f82 admin admin] PATH_INFO: `/v3/projects/189b4885ee8147d599f396c1bcd07f1d` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:41.080457 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c0ea2c-ae4f-490f-bd80-868fb8324f82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=189b4885ee8147d599f396c1bcd07f1d)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:41.082427 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c0ea2c-ae4f-490f-bd80-868fb8324f82 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:41.136898 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-45c0ea2c-ae4f-490f-bd80-868fb8324f82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '189b4885ee8147d599f396c1bcd07f1d', 'request_id': 'req-45c0ea2c-ae4f-490f-bd80-868fb8324f82'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:41.153752 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-45c0ea2c-ae4f-490f-bd80-868fb8324f82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '189b4885ee8147d599f396c1bcd07f1d', 'request_id': 'req-45c0ea2c-ae4f-490f-bd80-868fb8324f82'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:41.160410 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-45c0ea2c-ae4f-490f-bd80-868fb8324f82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '189b4885ee8147d599f396c1bcd07f1d', 'request_id': 'req-45c0ea2c-ae4f-490f-bd80-868fb8324f82'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:41.171721 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-45c0ea2c-ae4f-490f-bd80-868fb8324f82 admin admin] The token cache is being invalidate because project 189b4885ee8147d599f396c1bcd07f1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:03:41.172134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-45c0ea2c-ae4f-490f-bd80-868fb8324f82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:41.173061 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-45c0ea2c-ae4f-490f-bd80-868fb8324f82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:41.175640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1007/2029] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:03:40 2022] DELETE /identity/v3/projects/189b4885ee8147d599f396c1bcd07f1d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:03:41.179455 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319aa7b5-f65b-4b7f-92ae-a77587337086 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:41.181068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-319aa7b5-f65b-4b7f-92ae-a77587337086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:41.194288 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319aa7b5-f65b-4b7f-92ae-a77587337086 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:41.258399 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2be471-eacc-4e2c-8ebe-fd69991ca593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:41.259046 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2be471-eacc-4e2c-8ebe-fd69991ca593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:41.259329 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2be471-eacc-4e2c-8ebe-fd69991ca593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:41.259605 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2be471-eacc-4e2c-8ebe-fd69991ca593 admin admin] PATH_INFO: `/v3/users/84b52d77e8b14b92b7d1e6791a9d082a` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:41.260788 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2be471-eacc-4e2c-8ebe-fd69991ca593 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84b52d77e8b14b92b7d1e6791a9d082a)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:41.262712 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2be471-eacc-4e2c-8ebe-fd69991ca593 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:41.310381 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:41.314132 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-bd2be471-eacc-4e2c-8ebe-fd69991ca593 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84b52d77e8b14b92b7d1e6791a9d082a'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:41.330655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-bd2be471-eacc-4e2c-8ebe-fd69991ca593 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84b52d77e8b14b92b7d1e6791a9d082a'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:41.341690 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-bd2be471-eacc-4e2c-8ebe-fd69991ca593 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84b52d77e8b14b92b7d1e6791a9d082a'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:41.360057 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-bd2be471-eacc-4e2c-8ebe-fd69991ca593 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84b52d77e8b14b92b7d1e6791a9d082a'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:41.362011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-bd2be471-eacc-4e2c-8ebe-fd69991ca593 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84b52d77e8b14b92b7d1e6791a9d082a'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:41.368209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1023/2030] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:03:41 2022] DELETE /identity/v3/users/84b52d77e8b14b92b7d1e6791a9d082a => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:41.802195 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c0ea2c-ae4f-490f-bd80-868fb8324f82 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:41.803680 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-45c0ea2c-ae4f-490f-bd80-868fb8324f82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:41.827286 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c0ea2c-ae4f-490f-bd80-868fb8324f82 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:41.845130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:41.891404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a843de5-ea10-4714-b5bf-f97551696d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:41.892658 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a843de5-ea10-4714-b5bf-f97551696d2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:41.892949 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a843de5-ea10-4714-b5bf-f97551696d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:41.893227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a843de5-ea10-4714-b5bf-f97551696d2f admin admin] PATH_INFO: `/v3/projects/1f888c06fd33447492598cdb37c980eb` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:41.894391 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a843de5-ea10-4714-b5bf-f97551696d2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f888c06fd33447492598cdb37c980eb)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:41.896603 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a843de5-ea10-4714-b5bf-f97551696d2f admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:41.942576 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8a843de5-ea10-4714-b5bf-f97551696d2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f888c06fd33447492598cdb37c980eb', 'request_id': 'req-8a843de5-ea10-4714-b5bf-f97551696d2f'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:41.950751 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8a843de5-ea10-4714-b5bf-f97551696d2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f888c06fd33447492598cdb37c980eb', 'request_id': 'req-8a843de5-ea10-4714-b5bf-f97551696d2f'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:41.957621 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8a843de5-ea10-4714-b5bf-f97551696d2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f888c06fd33447492598cdb37c980eb', 'request_id': 'req-8a843de5-ea10-4714-b5bf-f97551696d2f'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:41.969765 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8a843de5-ea10-4714-b5bf-f97551696d2f admin admin] The token cache is being invalidate because project 1f888c06fd33447492598cdb37c980eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:03:41.970642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8a843de5-ea10-4714-b5bf-f97551696d2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:41.971465 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8a843de5-ea10-4714-b5bf-f97551696d2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:41.973043 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1008/2031] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:03:41 2022] DELETE /identity/v3/projects/1f888c06fd33447492598cdb37c980eb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:41.979202 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78e6ce-3f34-4215-ae90-18f18122f98f None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:41.979701 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78e6ce-3f34-4215-ae90-18f18122f98f None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:41.979794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78e6ce-3f34-4215-ae90-18f18122f98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:41.999362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-2b78e6ce-3f34-4215-ae90-18f18122f98f None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:42.000544 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-2b78e6ce-3f34-4215-ae90-18f18122f98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:42.065786 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1024/2032] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:42.074055 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9ebb5-c9c9-4234-947c-6e6492b2f9d0 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:42.074569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9ebb5-c9c9-4234-947c-6e6492b2f9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:42.074742 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9ebb5-c9c9-4234-947c-6e6492b2f9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:42.096364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-a7e9ebb5-c9c9-4234-947c-6e6492b2f9d0 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:42.097619 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-a7e9ebb5-c9c9-4234-947c-6e6492b2f9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:42.107119 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1009/2033] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:42.112284 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78e6ce-3f34-4215-ae90-18f18122f98f None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:42.124099 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78e6ce-3f34-4215-ae90-18f18122f98f None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:42.129472 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807a4c0-525d-44b0-bea1-76f2349c6dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:42.130094 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807a4c0-525d-44b0-bea1-76f2349c6dc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:42.130378 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807a4c0-525d-44b0-bea1-76f2349c6dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:42.130656 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807a4c0-525d-44b0-bea1-76f2349c6dc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:42.131357 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807a4c0-525d-44b0-bea1-76f2349c6dc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:03:42.131667 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807a4c0-525d-44b0-bea1-76f2349c6dc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:42.133319 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807a4c0-525d-44b0-bea1-76f2349c6dc2 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:42.147871 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1025/2034] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:03:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 20:03:42.152231 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e9ebb5-c9c9-4234-947c-6e6492b2f9d0 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:42.162447 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e9ebb5-c9c9-4234-947c-6e6492b2f9d0 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:42.167434 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd3f24-611a-4de0-b5a7-879772fb373c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:42.168051 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd3f24-611a-4de0-b5a7-879772fb373c admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:42.168363 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd3f24-611a-4de0-b5a7-879772fb373c admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:42.168744 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd3f24-611a-4de0-b5a7-879772fb373c admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:42.169426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd3f24-611a-4de0-b5a7-879772fb373c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:42.171342 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd3f24-611a-4de0-b5a7-879772fb373c admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:42.205527 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1010/2035] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:03:42 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:42.206743 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807a4c0-525d-44b0-bea1-76f2349c6dc2 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:42.217455 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807a4c0-525d-44b0-bea1-76f2349c6dc2 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:42.288366 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996d561-0f68-45bd-b371-51f12e39e1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:42.289026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996d561-0f68-45bd-b371-51f12e39e1c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:42.289314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996d561-0f68-45bd-b371-51f12e39e1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:42.289586 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996d561-0f68-45bd-b371-51f12e39e1c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:42.290256 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6996d561-0f68-45bd-b371-51f12e39e1c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:42.292242 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6996d561-0f68-45bd-b371-51f12e39e1c2 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:42.317629 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1026/2036] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:03:42 2022] POST /identity/v3/users => generated 291 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:03:42.321637 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd3f24-611a-4de0-b5a7-879772fb373c admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:42.334210 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd3f24-611a-4de0-b5a7-879772fb373c admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:42.340784 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ba408-3e52-4ab9-9de2-9dc95ceff01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:42.341625 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ba408-3e52-4ab9-9de2-9dc95ceff01e admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:42.342085 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ba408-3e52-4ab9-9de2-9dc95ceff01e admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:42.342384 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ba408-3e52-4ab9-9de2-9dc95ceff01e admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:42.343084 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ba408-3e52-4ab9-9de2-9dc95ceff01e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:42.347616 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ba408-3e52-4ab9-9de2-9dc95ceff01e admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:42.361324 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1011/2037] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:42.362848 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996d561-0f68-45bd-b371-51f12e39e1c2 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:42.373012 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996d561-0f68-45bd-b371-51f12e39e1c2 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:42.378835 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b3f21-a595-4090-b7ba-60da2f14b606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:42.379466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b3f21-a595-4090-b7ba-60da2f14b606 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:42.379783 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b3f21-a595-4090-b7ba-60da2f14b606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:42.380027 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b3f21-a595-4090-b7ba-60da2f14b606 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:42.380777 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b3f21-a595-4090-b7ba-60da2f14b606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:42.382608 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b3f21-a595-4090-b7ba-60da2f14b606 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:42.396620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1027/2038] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:42.399279 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ba408-3e52-4ab9-9de2-9dc95ceff01e admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:42.411693 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ba408-3e52-4ab9-9de2-9dc95ceff01e admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:42.417105 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700566fd-556a-4470-8cd3-46160e51f62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:42.417889 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700566fd-556a-4470-8cd3-46160e51f62b admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:42.418174 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700566fd-556a-4470-8cd3-46160e51f62b admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:42.418447 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700566fd-556a-4470-8cd3-46160e51f62b admin admin] PATH_INFO: `/v3/projects/7a5caf50650e45fbb1872e277f56e3b3/users/ac0a233c84e1413490057c64654abe99/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:42.434651 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700566fd-556a-4470-8cd3-46160e51f62b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a5caf50650e45fbb1872e277f56e3b3, user_id=ac0a233c84e1413490057c64654abe99, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:42.437321 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700566fd-556a-4470-8cd3-46160e51f62b admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:42.457350 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1012/2039] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:42 2022] PUT /identity/v3/projects/7a5caf50650e45fbb1872e277f56e3b3/users/ac0a233c84e1413490057c64654abe99/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:42.458837 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2f1d7-03f2-4422-923a-cd0664d25a62 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:42.459120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2f1d7-03f2-4422-923a-cd0664d25a62 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:42.459394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2f1d7-03f2-4422-923a-cd0664d25a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:42.479495 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-19c2f1d7-03f2-4422-923a-cd0664d25a62 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:42.480872 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-19c2f1d7-03f2-4422-923a-cd0664d25a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:42.509210 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:42.551405 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1028/2040] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:03:42.561287 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700566fd-556a-4470-8cd3-46160e51f62b admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:42.563000 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-700566fd-556a-4470-8cd3-46160e51f62b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:42.613709 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:42.638583 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700566fd-556a-4470-8cd3-46160e51f62b admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:42.645377 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4db8c-e694-4a28-8553-22a975f1913b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:42.646791 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4db8c-e694-4a28-8553-22a975f1913b service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:42.647195 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4db8c-e694-4a28-8553-22a975f1913b service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:42.647476 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4db8c-e694-4a28-8553-22a975f1913b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:42.660996 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4db8c-e694-4a28-8553-22a975f1913b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:42.662783 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4db8c-e694-4a28-8553-22a975f1913b service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:42.675401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1013/2041] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:03:43.927941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872a154-2fbc-4eca-8920-877ebe34a739 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:43.928635 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872a154-2fbc-4eca-8920-877ebe34a739 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:43.928635 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872a154-2fbc-4eca-8920-877ebe34a739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:43.984478 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-b872a154-2fbc-4eca-8920-877ebe34a739 None None] MFA Rules not processed for user `ac0a233c84e1413490057c64654abe99`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:43.987242 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-b872a154-2fbc-4eca-8920-877ebe34a739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:44.064499 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1029/2042] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:44.075131 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4db8c-e694-4a28-8553-22a975f1913b service neutron] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:44.077096 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-c8f4db8c-e694-4a28-8553-22a975f1913b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:44.149975 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4db8c-e694-4a28-8553-22a975f1913b service neutron] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:44.153802 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c9bfe-7ecc-4a25-91b8-2c0235b528a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:44.154417 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c9bfe-7ecc-4a25-91b8-2c0235b528a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:44.154700 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c9bfe-7ecc-4a25-91b8-2c0235b528a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:44.154976 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c9bfe-7ecc-4a25-91b8-2c0235b528a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:44.164781 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3c9bfe-7ecc-4a25-91b8-2c0235b528a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:44.166206 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3c9bfe-7ecc-4a25-91b8-2c0235b528a1 service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:44.175343 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1014/2043] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:44 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:03:51.888187 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449fe9f-a11d-4baa-ab88-904d08c7d6eb None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:51.888697 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449fe9f-a11d-4baa-ab88-904d08c7d6eb None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:51.888788 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449fe9f-a11d-4baa-ab88-904d08c7d6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:51.907946 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-b449fe9f-a11d-4baa-ab88-904d08c7d6eb None None] MFA Rules not processed for user `ac0a233c84e1413490057c64654abe99`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:51.909293 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-b449fe9f-a11d-4baa-ab88-904d08c7d6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:51.917142 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1030/2044] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:54.171138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=32 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:56.840016 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c9bfe-7ecc-4a25-91b8-2c0235b528a1 service cinder] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:56.841535 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-6e3c9bfe-7ecc-4a25-91b8-2c0235b528a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:56.856791 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c9bfe-7ecc-4a25-91b8-2c0235b528a1 service cinder] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:56.862174 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed53bbe0-2c13-49db-90e7-d68305c5ae7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:56.862817 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53bbe0-2c13-49db-90e7-d68305c5ae7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:56.863099 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53bbe0-2c13-49db-90e7-d68305c5ae7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:56.863404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53bbe0-2c13-49db-90e7-d68305c5ae7f admin admin] PATH_INFO: `/v3/users/a8253453c662476dbcb9c95f36bc3ad7` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:56.864659 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed53bbe0-2c13-49db-90e7-d68305c5ae7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8253453c662476dbcb9c95f36bc3ad7)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:56.866631 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed53bbe0-2c13-49db-90e7-d68305c5ae7f admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:56.915271 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ed53bbe0-2c13-49db-90e7-d68305c5ae7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8253453c662476dbcb9c95f36bc3ad7'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:56.926825 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ed53bbe0-2c13-49db-90e7-d68305c5ae7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8253453c662476dbcb9c95f36bc3ad7'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:56.946441 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ed53bbe0-2c13-49db-90e7-d68305c5ae7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8253453c662476dbcb9c95f36bc3ad7'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:56.947134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ed53bbe0-2c13-49db-90e7-d68305c5ae7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8253453c662476dbcb9c95f36bc3ad7'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:56.954690 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ed53bbe0-2c13-49db-90e7-d68305c5ae7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8253453c662476dbcb9c95f36bc3ad7'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:56.961455 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1015/2045] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:03:56 2022] DELETE /identity/v3/users/a8253453c662476dbcb9c95f36bc3ad7 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:56.965921 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aacca7-b546-4188-afc0-94184bba2244 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:56.966442 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aacca7-b546-4188-afc0-94184bba2244 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:56.966524 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aacca7-b546-4188-afc0-94184bba2244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:56.985937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-54aacca7-b546-4188-afc0-94184bba2244 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:56.987163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-54aacca7-b546-4188-afc0-94184bba2244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:57.048811 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1031/2046] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:57.056274 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed53bbe0-2c13-49db-90e7-d68305c5ae7f admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:57.057844 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-ed53bbe0-2c13-49db-90e7-d68305c5ae7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:57.115934 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed53bbe0-2c13-49db-90e7-d68305c5ae7f admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:57.119537 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc908bf7-f76d-49b1-adf2-6e29baea668c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:57.120179 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc908bf7-f76d-49b1-adf2-6e29baea668c service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:57.120456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc908bf7-f76d-49b1-adf2-6e29baea668c service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:57.120766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc908bf7-f76d-49b1-adf2-6e29baea668c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:57.130112 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc908bf7-f76d-49b1-adf2-6e29baea668c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:57.131513 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc908bf7-f76d-49b1-adf2-6e29baea668c service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:57.141299 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1016/2047] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:03:57.498613 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aacca7-b546-4188-afc0-94184bba2244 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:57.500297 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-54aacca7-b546-4188-afc0-94184bba2244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:57.519981 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aacca7-b546-4188-afc0-94184bba2244 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:57.538576 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034d8ea-8fe8-4c8d-8264-b47a52cae8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:57.539571 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034d8ea-8fe8-4c8d-8264-b47a52cae8de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:57.540220 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034d8ea-8fe8-4c8d-8264-b47a52cae8de admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:57.540851 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034d8ea-8fe8-4c8d-8264-b47a52cae8de admin admin] PATH_INFO: `/v3/projects/37778508d3ec49cab34cbbc5b222ba2b` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:57.544124 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c034d8ea-8fe8-4c8d-8264-b47a52cae8de admin admin] RBAC: Authorizing `identity:delete_project(project_id=37778508d3ec49cab34cbbc5b222ba2b)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:57.547213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c034d8ea-8fe8-4c8d-8264-b47a52cae8de admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:57.637722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:57.641420 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-c034d8ea-8fe8-4c8d-8264-b47a52cae8de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37778508d3ec49cab34cbbc5b222ba2b', 'request_id': 'req-c034d8ea-8fe8-4c8d-8264-b47a52cae8de'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:57.650891 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-c034d8ea-8fe8-4c8d-8264-b47a52cae8de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37778508d3ec49cab34cbbc5b222ba2b', 'request_id': 'req-c034d8ea-8fe8-4c8d-8264-b47a52cae8de'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:57.659995 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-c034d8ea-8fe8-4c8d-8264-b47a52cae8de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37778508d3ec49cab34cbbc5b222ba2b', 'request_id': 'req-c034d8ea-8fe8-4c8d-8264-b47a52cae8de'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:57.678022 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-c034d8ea-8fe8-4c8d-8264-b47a52cae8de admin admin] The token cache is being invalidate because project 37778508d3ec49cab34cbbc5b222ba2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:03:57.678422 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-c034d8ea-8fe8-4c8d-8264-b47a52cae8de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:57.679151 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-c034d8ea-8fe8-4c8d-8264-b47a52cae8de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:03:57.680834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1032/2048] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:03:57 2022] DELETE /identity/v3/projects/37778508d3ec49cab34cbbc5b222ba2b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:57.685724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078819e-a860-4160-81dd-6a504031d001 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:57.685998 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078819e-a860-4160-81dd-6a504031d001 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:57.686275 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078819e-a860-4160-81dd-6a504031d001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:57.707861 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-f078819e-a860-4160-81dd-6a504031d001 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:57.709097 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-f078819e-a860-4160-81dd-6a504031d001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:57.775766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1017/2049] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:57.784303 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68295125-21e1-419b-8d1b-19d59bd7cc5c None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:57.784621 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68295125-21e1-419b-8d1b-19d59bd7cc5c None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:57.784904 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68295125-21e1-419b-8d1b-19d59bd7cc5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:57.804467 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-68295125-21e1-419b-8d1b-19d59bd7cc5c None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:57.805635 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-68295125-21e1-419b-8d1b-19d59bd7cc5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:57.814058 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1033/2050] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:57.818270 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078819e-a860-4160-81dd-6a504031d001 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:57.828336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078819e-a860-4160-81dd-6a504031d001 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:57.833615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49395df5-cec6-440f-95b1-95097945bfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:57.834217 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49395df5-cec6-440f-95b1-95097945bfba admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:57.834497 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49395df5-cec6-440f-95b1-95097945bfba admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:57.834771 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49395df5-cec6-440f-95b1-95097945bfba admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:57.835461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49395df5-cec6-440f-95b1-95097945bfba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:03:57.835768 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49395df5-cec6-440f-95b1-95097945bfba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:57.837373 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49395df5-cec6-440f-95b1-95097945bfba admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:57.848957 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1018/2051] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:03:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:03:57.853177 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68295125-21e1-419b-8d1b-19d59bd7cc5c None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:57.863126 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68295125-21e1-419b-8d1b-19d59bd7cc5c None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:57.867776 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45bac50-9587-4588-9bdf-a5d42efff624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:57.868370 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45bac50-9587-4588-9bdf-a5d42efff624 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:57.868717 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45bac50-9587-4588-9bdf-a5d42efff624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:57.868994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45bac50-9587-4588-9bdf-a5d42efff624 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:57.869658 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45bac50-9587-4588-9bdf-a5d42efff624 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:57.871554 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45bac50-9587-4588-9bdf-a5d42efff624 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:57.899069 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1034/2052] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:03:57 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:03:57.903307 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49395df5-cec6-440f-95b1-95097945bfba admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:57.909280 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:03:57.915820 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49395df5-cec6-440f-95b1-95097945bfba admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:57.972581 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a3c23d-7d0b-48b8-b040-fa6e7cf86e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:57.973210 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3c23d-7d0b-48b8-b040-fa6e7cf86e74 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:57.973486 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3c23d-7d0b-48b8-b040-fa6e7cf86e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:57.973759 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3c23d-7d0b-48b8-b040-fa6e7cf86e74 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:57.974414 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a3c23d-7d0b-48b8-b040-fa6e7cf86e74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:57.976298 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a3c23d-7d0b-48b8-b040-fa6e7cf86e74 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:57.997597 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1019/2053] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:03:57 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:03:58.002441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45bac50-9587-4588-9bdf-a5d42efff624 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:58.012987 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45bac50-9587-4588-9bdf-a5d42efff624 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:58.017571 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a6d62-8979-4ece-84f7-0ff4e2adad35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:58.018171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a6d62-8979-4ece-84f7-0ff4e2adad35 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:58.018452 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a6d62-8979-4ece-84f7-0ff4e2adad35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:58.018727 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a6d62-8979-4ece-84f7-0ff4e2adad35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:58.019412 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a6d62-8979-4ece-84f7-0ff4e2adad35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:58.021314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a6d62-8979-4ece-84f7-0ff4e2adad35 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:58.031262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1035/2054] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:58.035428 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a3c23d-7d0b-48b8-b040-fa6e7cf86e74 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:58.045124 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a3c23d-7d0b-48b8-b040-fa6e7cf86e74 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:58.050293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bbd78a-7cbe-4733-b7e5-c34122ffade9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:58.050887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbd78a-7cbe-4733-b7e5-c34122ffade9 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:58.051162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbd78a-7cbe-4733-b7e5-c34122ffade9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:58.051431 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbd78a-7cbe-4733-b7e5-c34122ffade9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:58.052114 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bbd78a-7cbe-4733-b7e5-c34122ffade9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:58.053708 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bbd78a-7cbe-4733-b7e5-c34122ffade9 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:58.065275 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1020/2055] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:03:58.069314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a6d62-8979-4ece-84f7-0ff4e2adad35 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:58.079205 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a6d62-8979-4ece-84f7-0ff4e2adad35 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:58.083815 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155445d-4827-4507-8c9c-6795245b8ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:58.084467 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155445d-4827-4507-8c9c-6795245b8ae5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:58.084781 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155445d-4827-4507-8c9c-6795245b8ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:58.085062 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155445d-4827-4507-8c9c-6795245b8ae5 admin admin] PATH_INFO: `/v3/projects/87bd21f57497428cb37b5669b7679b2a/users/478c4a45edad412c97680ed239931e96/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:58.099317 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3155445d-4827-4507-8c9c-6795245b8ae5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87bd21f57497428cb37b5669b7679b2a, user_id=478c4a45edad412c97680ed239931e96, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:58.102003 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3155445d-4827-4507-8c9c-6795245b8ae5 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:58.116019 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1036/2056] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:03:58 2022] PUT /identity/v3/projects/87bd21f57497428cb37b5669b7679b2a/users/478c4a45edad412c97680ed239931e96/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:03:58.128120 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d46fa0-1c93-4652-aeaf-edca16addfc0 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:58.128408 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d46fa0-1c93-4652-aeaf-edca16addfc0 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:58.128724 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d46fa0-1c93-4652-aeaf-edca16addfc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:58.157988 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-83d46fa0-1c93-4652-aeaf-edca16addfc0 None None] MFA Rules not processed for user `478c4a45edad412c97680ed239931e96`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:03:58.159199 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-83d46fa0-1c93-4652-aeaf-edca16addfc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:58.223129 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1021/2057] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:03:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:03:58.238963 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155445d-4827-4507-8c9c-6795245b8ae5 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:03:58.240712 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-3155445d-4827-4507-8c9c-6795245b8ae5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:03:58.297303 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155445d-4827-4507-8c9c-6795245b8ae5 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:03:58.301148 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0305f17-01c4-4aee-85c0-e9211f9ce155 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:03:58.301787 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0305f17-01c4-4aee-85c0-e9211f9ce155 service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:03:58.302096 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0305f17-01c4-4aee-85c0-e9211f9ce155 service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:03:58.302374 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0305f17-01c4-4aee-85c0-e9211f9ce155 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:03:58.310543 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0305f17-01c4-4aee-85c0-e9211f9ce155 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:03:58.312006 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0305f17-01c4-4aee-85c0-e9211f9ce155 service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:03:58.321449 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1037/2058] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:03:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:04:08.219030 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=65,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:08.317441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:DELETE=4,keystone:SELECT=60,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:12.118412 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d46fa0-1c93-4652-aeaf-edca16addfc0 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:12.119912 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-83d46fa0-1c93-4652-aeaf-edca16addfc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:12.286146 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d46fa0-1c93-4652-aeaf-edca16addfc0 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:12.297089 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b3dd2-5820-4b8e-960f-cab00d686048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:12.311727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b3dd2-5820-4b8e-960f-cab00d686048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:12.321998 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b3dd2-5820-4b8e-960f-cab00d686048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:12.323201 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b3dd2-5820-4b8e-960f-cab00d686048 admin admin] PATH_INFO: `/v3/users/2a26ee339ae54dc0811ed24400fa9e79` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:12.329798 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b3dd2-5820-4b8e-960f-cab00d686048 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a26ee339ae54dc0811ed24400fa9e79)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:12.332150 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b3dd2-5820-4b8e-960f-cab00d686048 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:12.405453 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-d21b3dd2-5820-4b8e-960f-cab00d686048 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a26ee339ae54dc0811ed24400fa9e79'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:12.415129 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-d21b3dd2-5820-4b8e-960f-cab00d686048 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a26ee339ae54dc0811ed24400fa9e79'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:12.430932 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-d21b3dd2-5820-4b8e-960f-cab00d686048 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a26ee339ae54dc0811ed24400fa9e79'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:12.432660 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-d21b3dd2-5820-4b8e-960f-cab00d686048 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a26ee339ae54dc0811ed24400fa9e79'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:12.446037 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-d21b3dd2-5820-4b8e-960f-cab00d686048 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a26ee339ae54dc0811ed24400fa9e79'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:12.456618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1022/2059] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:04:12 2022] DELETE /identity/v3/users/2a26ee339ae54dc0811ed24400fa9e79 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:04:12.667734 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0305f17-01c4-4aee-85c0-e9211f9ce155 service cinder] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:12.681272 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0305f17-01c4-4aee-85c0-e9211f9ce155 service cinder] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:12.750047 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ee023-bb20-48a9-9e61-2beda009ecb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:12.750951 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ee023-bb20-48a9-9e61-2beda009ecb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:12.751366 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ee023-bb20-48a9-9e61-2beda009ecb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:12.751654 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ee023-bb20-48a9-9e61-2beda009ecb2 admin admin] PATH_INFO: `/v3/projects/08cadf2cf9de44cf8f6cdd58dcb717c5` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:12.752972 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ee023-bb20-48a9-9e61-2beda009ecb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08cadf2cf9de44cf8f6cdd58dcb717c5)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:12.754895 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ee023-bb20-48a9-9e61-2beda009ecb2 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:12.807414 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-602ee023-bb20-48a9-9e61-2beda009ecb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08cadf2cf9de44cf8f6cdd58dcb717c5', 'request_id': 'req-602ee023-bb20-48a9-9e61-2beda009ecb2'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:12.821962 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-602ee023-bb20-48a9-9e61-2beda009ecb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08cadf2cf9de44cf8f6cdd58dcb717c5', 'request_id': 'req-602ee023-bb20-48a9-9e61-2beda009ecb2'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:12.837134 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-602ee023-bb20-48a9-9e61-2beda009ecb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08cadf2cf9de44cf8f6cdd58dcb717c5', 'request_id': 'req-602ee023-bb20-48a9-9e61-2beda009ecb2'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:12.853091 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-602ee023-bb20-48a9-9e61-2beda009ecb2 admin admin] The token cache is being invalidate because project 08cadf2cf9de44cf8f6cdd58dcb717c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:04:12.853396 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-602ee023-bb20-48a9-9e61-2beda009ecb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:12.854146 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-602ee023-bb20-48a9-9e61-2beda009ecb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:12.855952 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1038/2060] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:04:12 2022] DELETE /identity/v3/projects/08cadf2cf9de44cf8f6cdd58dcb717c5 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:04:12.861809 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b3dd2-5820-4b8e-960f-cab00d686048 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:12.863649 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-d21b3dd2-5820-4b8e-960f-cab00d686048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:12.878148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b3dd2-5820-4b8e-960f-cab00d686048 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:12.980936 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e25f1a-a30d-4f68-84d7-665c4ee69de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:12.981846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e25f1a-a30d-4f68-84d7-665c4ee69de5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:12.982230 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e25f1a-a30d-4f68-84d7-665c4ee69de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:12.982612 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e25f1a-a30d-4f68-84d7-665c4ee69de5 admin admin] PATH_INFO: `/v3/users/bc6a15df3d39474f94addf30c4ed9d63` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:12.984754 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e25f1a-a30d-4f68-84d7-665c4ee69de5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc6a15df3d39474f94addf30c4ed9d63)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:12.987118 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e25f1a-a30d-4f68-84d7-665c4ee69de5 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:13.013878 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:13.044334 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-50e25f1a-a30d-4f68-84d7-665c4ee69de5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc6a15df3d39474f94addf30c4ed9d63'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:13.053919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-50e25f1a-a30d-4f68-84d7-665c4ee69de5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc6a15df3d39474f94addf30c4ed9d63'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:13.075106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-50e25f1a-a30d-4f68-84d7-665c4ee69de5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc6a15df3d39474f94addf30c4ed9d63'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:13.075880 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-50e25f1a-a30d-4f68-84d7-665c4ee69de5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc6a15df3d39474f94addf30c4ed9d63'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:13.087814 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-50e25f1a-a30d-4f68-84d7-665c4ee69de5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc6a15df3d39474f94addf30c4ed9d63'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:13.094972 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1023/2061] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:04:12 2022] DELETE /identity/v3/users/bc6a15df3d39474f94addf30c4ed9d63 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:04:13.294538 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ee023-bb20-48a9-9e61-2beda009ecb2 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:13.298281 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-602ee023-bb20-48a9-9e61-2beda009ecb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:13.314029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ee023-bb20-48a9-9e61-2beda009ecb2 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:13.382812 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9cdfb0-03e0-4b9f-ad0a-e58c15bc1e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:13.383465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9cdfb0-03e0-4b9f-ad0a-e58c15bc1e9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:13.383742 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9cdfb0-03e0-4b9f-ad0a-e58c15bc1e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:13.384012 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9cdfb0-03e0-4b9f-ad0a-e58c15bc1e9c admin admin] PATH_INFO: `/v3/projects/cb0905767c48451494dddf9c3008edf5` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:13.385202 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9cdfb0-03e0-4b9f-ad0a-e58c15bc1e9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb0905767c48451494dddf9c3008edf5)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:13.387115 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9cdfb0-03e0-4b9f-ad0a-e58c15bc1e9c admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:13.434474 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:13.460063 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-9b9cdfb0-03e0-4b9f-ad0a-e58c15bc1e9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb0905767c48451494dddf9c3008edf5', 'request_id': 'req-9b9cdfb0-03e0-4b9f-ad0a-e58c15bc1e9c'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:13.464282 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-9b9cdfb0-03e0-4b9f-ad0a-e58c15bc1e9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb0905767c48451494dddf9c3008edf5', 'request_id': 'req-9b9cdfb0-03e0-4b9f-ad0a-e58c15bc1e9c'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:13.494017 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-9b9cdfb0-03e0-4b9f-ad0a-e58c15bc1e9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb0905767c48451494dddf9c3008edf5', 'request_id': 'req-9b9cdfb0-03e0-4b9f-ad0a-e58c15bc1e9c'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:13.525232 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-9b9cdfb0-03e0-4b9f-ad0a-e58c15bc1e9c admin admin] The token cache is being invalidate because project cb0905767c48451494dddf9c3008edf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:04:13.525528 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-9b9cdfb0-03e0-4b9f-ad0a-e58c15bc1e9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:13.526432 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-9b9cdfb0-03e0-4b9f-ad0a-e58c15bc1e9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:13.529014 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1039/2062] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:04:13 2022] DELETE /identity/v3/projects/cb0905767c48451494dddf9c3008edf5 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:04:13.535483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f770686-e2f1-4e28-bd3e-920fbfcff0e1 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:13.536523 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f770686-e2f1-4e28-bd3e-920fbfcff0e1 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:13.536791 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f770686-e2f1-4e28-bd3e-920fbfcff0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:13.575554 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-5f770686-e2f1-4e28-bd3e-920fbfcff0e1 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:04:13.587204 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-5f770686-e2f1-4e28-bd3e-920fbfcff0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:13.681842 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1024/2063] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:04:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:04:13.690603 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8600c88-9bb3-48a3-9851-0cc3873af8bd None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:13.691138 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8600c88-9bb3-48a3-9851-0cc3873af8bd None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:13.691338 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8600c88-9bb3-48a3-9851-0cc3873af8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:13.720207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-e8600c88-9bb3-48a3-9851-0cc3873af8bd None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:04:13.721653 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e8600c88-9bb3-48a3-9851-0cc3873af8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:13.730360 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1040/2064] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:04:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:04:13.737810 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f770686-e2f1-4e28-bd3e-920fbfcff0e1 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:13.753496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f770686-e2f1-4e28-bd3e-920fbfcff0e1 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:13.758515 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7cd2b-8ba8-4ba6-a51b-114f2f2327df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:13.759143 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7cd2b-8ba8-4ba6-a51b-114f2f2327df admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:13.759426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7cd2b-8ba8-4ba6-a51b-114f2f2327df admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:13.759704 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7cd2b-8ba8-4ba6-a51b-114f2f2327df admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:13.761183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf7cd2b-8ba8-4ba6-a51b-114f2f2327df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:04:13.761555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf7cd2b-8ba8-4ba6-a51b-114f2f2327df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:13.763909 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf7cd2b-8ba8-4ba6-a51b-114f2f2327df admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:13.781671 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1025/2065] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:04:13.782798 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8600c88-9bb3-48a3-9851-0cc3873af8bd None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:13.792266 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8600c88-9bb3-48a3-9851-0cc3873af8bd None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:13.799582 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a1d77-4ed0-448d-be97-d29114a6aebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:13.800543 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a1d77-4ed0-448d-be97-d29114a6aebd admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:13.801003 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a1d77-4ed0-448d-be97-d29114a6aebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:13.801417 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a1d77-4ed0-448d-be97-d29114a6aebd admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:13.802262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9a1d77-4ed0-448d-be97-d29114a6aebd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:13.804354 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9a1d77-4ed0-448d-be97-d29114a6aebd admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:13.838583 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1041/2066] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:04:13 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:04:13.842547 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7cd2b-8ba8-4ba6-a51b-114f2f2327df admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:13.855383 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7cd2b-8ba8-4ba6-a51b-114f2f2327df admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:13.915982 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a1345-0bea-4fc5-a03c-d0928d042195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:13.916663 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a1345-0bea-4fc5-a03c-d0928d042195 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:13.916946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a1345-0bea-4fc5-a03c-d0928d042195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:13.917222 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a1345-0bea-4fc5-a03c-d0928d042195 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:13.917943 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a1345-0bea-4fc5-a03c-d0928d042195 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:13.920305 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a1345-0bea-4fc5-a03c-d0928d042195 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:13.948030 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1026/2067] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:04:13 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:04:13.949357 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a1d77-4ed0-448d-be97-d29114a6aebd admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:13.960085 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a1d77-4ed0-448d-be97-d29114a6aebd admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:13.964785 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313543d9-ea6f-446c-bc0b-2292e59b24b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:13.965397 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313543d9-ea6f-446c-bc0b-2292e59b24b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:13.965675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313543d9-ea6f-446c-bc0b-2292e59b24b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:13.966046 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313543d9-ea6f-446c-bc0b-2292e59b24b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:13.966765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313543d9-ea6f-446c-bc0b-2292e59b24b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:13.969045 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313543d9-ea6f-446c-bc0b-2292e59b24b2 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:13.979684 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1042/2068] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:04:13.983677 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a1345-0bea-4fc5-a03c-d0928d042195 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:13.993876 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a1345-0bea-4fc5-a03c-d0928d042195 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:13.998575 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e03f86-2b8c-4502-94fa-adcb070a070b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:13.999941 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e03f86-2b8c-4502-94fa-adcb070a070b admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:14.000080 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e03f86-2b8c-4502-94fa-adcb070a070b admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:14.000341 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e03f86-2b8c-4502-94fa-adcb070a070b admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:14.001114 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e03f86-2b8c-4502-94fa-adcb070a070b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:14.002819 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e03f86-2b8c-4502-94fa-adcb070a070b admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:14.017443 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1027/2069] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:04:14.021889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313543d9-ea6f-446c-bc0b-2292e59b24b2 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:14.032622 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313543d9-ea6f-446c-bc0b-2292e59b24b2 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:14.037248 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae07a9-3a34-41f6-a1b8-693a83cec230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:14.037909 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae07a9-3a34-41f6-a1b8-693a83cec230 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:14.038183 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae07a9-3a34-41f6-a1b8-693a83cec230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:14.038456 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae07a9-3a34-41f6-a1b8-693a83cec230 admin admin] PATH_INFO: `/v3/projects/53c17d51b00349ebb85dec271e1e1d0a/users/fce3be14f5db499baa38eaa46bf1c4f4/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:14.054775 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae07a9-3a34-41f6-a1b8-693a83cec230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53c17d51b00349ebb85dec271e1e1d0a, user_id=fce3be14f5db499baa38eaa46bf1c4f4, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:14.057661 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae07a9-3a34-41f6-a1b8-693a83cec230 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:14.071387 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1043/2070] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:04:14 2022] PUT /identity/v3/projects/53c17d51b00349ebb85dec271e1e1d0a/users/fce3be14f5db499baa38eaa46bf1c4f4/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:04:14.082851 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cc55d-b97e-4548-997f-659f7eaeb1fd None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:14.083165 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cc55d-b97e-4548-997f-659f7eaeb1fd None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:14.083448 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cc55d-b97e-4548-997f-659f7eaeb1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:14.091474 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:14.113853 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-c04cc55d-b97e-4548-997f-659f7eaeb1fd None None] MFA Rules not processed for user `fce3be14f5db499baa38eaa46bf1c4f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:04:14.115057 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-c04cc55d-b97e-4548-997f-659f7eaeb1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:14.183606 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1028/2071] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:04:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:04:14.194555 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae07a9-3a34-41f6-a1b8-693a83cec230 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:14.196351 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e5ae07a9-3a34-41f6-a1b8-693a83cec230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:14.254205 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae07a9-3a34-41f6-a1b8-693a83cec230 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:14.257842 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed37dec-b558-4af1-a26d-929a9971b1d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:14.258447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed37dec-b558-4af1-a26d-929a9971b1d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:14.258726 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed37dec-b558-4af1-a26d-929a9971b1d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:14.259025 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed37dec-b558-4af1-a26d-929a9971b1d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:14.268285 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed37dec-b558-4af1-a26d-929a9971b1d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:14.270441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed37dec-b558-4af1-a26d-929a9971b1d5 service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:14.279319 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1044/2072] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:04:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:04:24.180791 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=30 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:24.275252 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=65,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:40.700024 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04cc55d-b97e-4548-997f-659f7eaeb1fd None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:40.769349 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04cc55d-b97e-4548-997f-659f7eaeb1fd None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:40.780023 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fff55a-7a30-4457-b41b-3780e5c84cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:40.780023 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fff55a-7a30-4457-b41b-3780e5c84cc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:40.780023 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fff55a-7a30-4457-b41b-3780e5c84cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:40.780023 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fff55a-7a30-4457-b41b-3780e5c84cc6 admin admin] PATH_INFO: `/v3/users/fce3be14f5db499baa38eaa46bf1c4f4` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:40.781035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fff55a-7a30-4457-b41b-3780e5c84cc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fce3be14f5db499baa38eaa46bf1c4f4)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:40.781035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fff55a-7a30-4457-b41b-3780e5c84cc6 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:40.851693 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-05fff55a-7a30-4457-b41b-3780e5c84cc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fce3be14f5db499baa38eaa46bf1c4f4'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:40.857512 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-05fff55a-7a30-4457-b41b-3780e5c84cc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fce3be14f5db499baa38eaa46bf1c4f4'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:40.870342 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-05fff55a-7a30-4457-b41b-3780e5c84cc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fce3be14f5db499baa38eaa46bf1c4f4'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:40.871026 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-05fff55a-7a30-4457-b41b-3780e5c84cc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fce3be14f5db499baa38eaa46bf1c4f4'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:40.878789 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-05fff55a-7a30-4457-b41b-3780e5c84cc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fce3be14f5db499baa38eaa46bf1c4f4'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:40.885432 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1029/2073] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:04:40 2022] DELETE /identity/v3/users/fce3be14f5db499baa38eaa46bf1c4f4 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:04:40.890346 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ed8eb-1a61-4155-83a3-e81260a1d034 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:40.890854 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ed8eb-1a61-4155-83a3-e81260a1d034 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:40.890854 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ed8eb-1a61-4155-83a3-e81260a1d034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:40.911347 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-5c9ed8eb-1a61-4155-83a3-e81260a1d034 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:04:40.912621 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5c9ed8eb-1a61-4155-83a3-e81260a1d034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:40.987274 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1045/2074] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:04:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:04:41.002090 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fff55a-7a30-4457-b41b-3780e5c84cc6 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:41.003892 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-05fff55a-7a30-4457-b41b-3780e5c84cc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:41.121095 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fff55a-7a30-4457-b41b-3780e5c84cc6 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:41.126509 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1779b9f-6972-4f3b-9db7-b5f16073cdb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:41.127140 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1779b9f-6972-4f3b-9db7-b5f16073cdb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:41.127422 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1779b9f-6972-4f3b-9db7-b5f16073cdb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:41.127698 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1779b9f-6972-4f3b-9db7-b5f16073cdb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:41.137408 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1779b9f-6972-4f3b-9db7-b5f16073cdb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:41.139589 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1779b9f-6972-4f3b-9db7-b5f16073cdb4 service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:41.151739 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1030/2075] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:04:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:04:41.728561 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ed8eb-1a61-4155-83a3-e81260a1d034 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:41.730188 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5c9ed8eb-1a61-4155-83a3-e81260a1d034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:41.743281 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ed8eb-1a61-4155-83a3-e81260a1d034 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:41.749432 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c1647-d27c-4c39-9673-a2efa81d590f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:41.750464 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c1647-d27c-4c39-9673-a2efa81d590f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:41.750793 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c1647-d27c-4c39-9673-a2efa81d590f admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:41.751185 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c1647-d27c-4c39-9673-a2efa81d590f admin admin] PATH_INFO: `/v3/projects/53c17d51b00349ebb85dec271e1e1d0a` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:41.752528 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c1647-d27c-4c39-9673-a2efa81d590f admin admin] RBAC: Authorizing `identity:delete_project(project_id=53c17d51b00349ebb85dec271e1e1d0a)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:41.757368 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c1647-d27c-4c39-9673-a2efa81d590f admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:41.810010 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-9e0c1647-d27c-4c39-9673-a2efa81d590f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53c17d51b00349ebb85dec271e1e1d0a', 'request_id': 'req-9e0c1647-d27c-4c39-9673-a2efa81d590f'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:41.819745 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-9e0c1647-d27c-4c39-9673-a2efa81d590f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53c17d51b00349ebb85dec271e1e1d0a', 'request_id': 'req-9e0c1647-d27c-4c39-9673-a2efa81d590f'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:41.829557 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-9e0c1647-d27c-4c39-9673-a2efa81d590f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53c17d51b00349ebb85dec271e1e1d0a', 'request_id': 'req-9e0c1647-d27c-4c39-9673-a2efa81d590f'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:41.842605 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-9e0c1647-d27c-4c39-9673-a2efa81d590f admin admin] The token cache is being invalidate because project 53c17d51b00349ebb85dec271e1e1d0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:04:41.842896 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-9e0c1647-d27c-4c39-9673-a2efa81d590f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:41.843689 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-9e0c1647-d27c-4c39-9673-a2efa81d590f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:41.845474 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1046/2076] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:04:41 2022] DELETE /identity/v3/projects/53c17d51b00349ebb85dec271e1e1d0a => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:04:41.850750 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd76a4-d029-4c63-b856-33023e7e2be2 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:41.851292 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd76a4-d029-4c63-b856-33023e7e2be2 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:41.851440 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd76a4-d029-4c63-b856-33023e7e2be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:41.873545 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-59dd76a4-d029-4c63-b856-33023e7e2be2 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:04:41.874840 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-59dd76a4-d029-4c63-b856-33023e7e2be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:41.895513 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:41.947971 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1031/2077] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:04:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:04:41.957784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbb58d-042f-42ef-bf54-b0173551cb90 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:41.958265 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbb58d-042f-42ef-bf54-b0173551cb90 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:41.958409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbb58d-042f-42ef-bf54-b0173551cb90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:41.979900 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-defbb58d-042f-42ef-bf54-b0173551cb90 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:04:41.981142 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-defbb58d-042f-42ef-bf54-b0173551cb90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:41.990310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1047/2078] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:04:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:04:41.995022 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dd76a4-d029-4c63-b856-33023e7e2be2 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:42.006178 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dd76a4-d029-4c63-b856-33023e7e2be2 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:42.011020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f0ba1-f063-4972-b968-9008be91e3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:42.011741 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f0ba1-f063-4972-b968-9008be91e3bb admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:42.012097 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f0ba1-f063-4972-b968-9008be91e3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:42.012456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f0ba1-f063-4972-b968-9008be91e3bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:42.013272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f0ba1-f063-4972-b968-9008be91e3bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:04:42.013702 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f0ba1-f063-4972-b968-9008be91e3bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:42.016853 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f0ba1-f063-4972-b968-9008be91e3bb admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:42.032812 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1032/2079] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:04:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:04:42.037189 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defbb58d-042f-42ef-bf54-b0173551cb90 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:42.049520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defbb58d-042f-42ef-bf54-b0173551cb90 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:42.058089 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817949c-04f9-41ce-92a5-805b29ee559c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:42.059158 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817949c-04f9-41ce-92a5-805b29ee559c admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:42.059753 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817949c-04f9-41ce-92a5-805b29ee559c admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:42.060794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817949c-04f9-41ce-92a5-805b29ee559c admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:42.064423 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4817949c-04f9-41ce-92a5-805b29ee559c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:42.066976 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4817949c-04f9-41ce-92a5-805b29ee559c admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:42.126340 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1048/2080] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:04:42 2022] POST /identity/v3/projects => generated 375 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:04:42.130803 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f0ba1-f063-4972-b968-9008be91e3bb admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:42.153116 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f0ba1-f063-4972-b968-9008be91e3bb admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:42.252294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01fb70a-f2c7-47d5-9991-dbbb685a9308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:42.253042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01fb70a-f2c7-47d5-9991-dbbb685a9308 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:42.253462 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01fb70a-f2c7-47d5-9991-dbbb685a9308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:42.254220 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01fb70a-f2c7-47d5-9991-dbbb685a9308 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:42.254541 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01fb70a-f2c7-47d5-9991-dbbb685a9308 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:42.256594 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01fb70a-f2c7-47d5-9991-dbbb685a9308 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:42.307585 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817949c-04f9-41ce-92a5-805b29ee559c admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:42.308001 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1033/2081] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:04:42 2022] POST /identity/v3/users => generated 288 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:04:42.323997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817949c-04f9-41ce-92a5-805b29ee559c admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:42.337870 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a03ff-dae8-4a0c-81a2-08cc8a6facc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:42.338509 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a03ff-dae8-4a0c-81a2-08cc8a6facc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:42.338791 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a03ff-dae8-4a0c-81a2-08cc8a6facc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:42.339062 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a03ff-dae8-4a0c-81a2-08cc8a6facc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:42.339968 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09a03ff-dae8-4a0c-81a2-08cc8a6facc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:42.342163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09a03ff-dae8-4a0c-81a2-08cc8a6facc7 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:42.354838 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1049/2082] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:04:42.359127 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01fb70a-f2c7-47d5-9991-dbbb685a9308 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:42.373208 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01fb70a-f2c7-47d5-9991-dbbb685a9308 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:42.384314 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f71fb3-3aa2-4bdd-991d-75274f6fc866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:42.384994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f71fb3-3aa2-4bdd-991d-75274f6fc866 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:42.385518 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f71fb3-3aa2-4bdd-991d-75274f6fc866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:42.385518 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f71fb3-3aa2-4bdd-991d-75274f6fc866 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:42.386255 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f71fb3-3aa2-4bdd-991d-75274f6fc866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:42.391907 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f71fb3-3aa2-4bdd-991d-75274f6fc866 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:42.406943 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1034/2083] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:04:42.413960 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a03ff-dae8-4a0c-81a2-08cc8a6facc7 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:42.434548 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a03ff-dae8-4a0c-81a2-08cc8a6facc7 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:42.441272 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4053ea5-ac56-4aaa-a34e-9da425e79796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:42.442229 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4053ea5-ac56-4aaa-a34e-9da425e79796 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:42.442737 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4053ea5-ac56-4aaa-a34e-9da425e79796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:42.443206 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4053ea5-ac56-4aaa-a34e-9da425e79796 admin admin] PATH_INFO: `/v3/projects/64d5d57427914355aadd94f85db396bd/users/4e575ea70ead4e4dbae3a1589b3898be/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:42.478695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4053ea5-ac56-4aaa-a34e-9da425e79796 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64d5d57427914355aadd94f85db396bd, user_id=4e575ea70ead4e4dbae3a1589b3898be, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:42.483154 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4053ea5-ac56-4aaa-a34e-9da425e79796 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:42.499084 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1050/2084] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:04:42 2022] PUT /identity/v3/projects/64d5d57427914355aadd94f85db396bd/users/4e575ea70ead4e4dbae3a1589b3898be/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:04:42.509885 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298aa2f6-2ea2-447c-8bc8-78a453ce348d None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:42.510647 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298aa2f6-2ea2-447c-8bc8-78a453ce348d None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:42.510856 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298aa2f6-2ea2-447c-8bc8-78a453ce348d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:42.546411 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-298aa2f6-2ea2-447c-8bc8-78a453ce348d None None] MFA Rules not processed for user `4e575ea70ead4e4dbae3a1589b3898be`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:04:42.548286 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-298aa2f6-2ea2-447c-8bc8-78a453ce348d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:42.651771 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1035/2085] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:04:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:04:42.659698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4053ea5-ac56-4aaa-a34e-9da425e79796 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:42.661748 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-f4053ea5-ac56-4aaa-a34e-9da425e79796 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:42.697812 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:42.743941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4053ea5-ac56-4aaa-a34e-9da425e79796 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:42.769006 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af88042-a2e7-4fac-af61-9faf1279319f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:42.769660 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af88042-a2e7-4fac-af61-9faf1279319f service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:42.769941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af88042-a2e7-4fac-af61-9faf1279319f service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:42.770212 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af88042-a2e7-4fac-af61-9faf1279319f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:42.785518 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af88042-a2e7-4fac-af61-9faf1279319f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:42.787191 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af88042-a2e7-4fac-af61-9faf1279319f service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:42.799337 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1051/2086] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:04:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:04:43.441881 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298aa2f6-2ea2-447c-8bc8-78a453ce348d None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:43.443359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-298aa2f6-2ea2-447c-8bc8-78a453ce348d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:43.468226 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:43.552128 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298aa2f6-2ea2-447c-8bc8-78a453ce348d None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:43.562955 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a15cfc-cc80-4d14-ab0e-f870962a0480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:43.565212 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a15cfc-cc80-4d14-ab0e-f870962a0480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:43.566245 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a15cfc-cc80-4d14-ab0e-f870962a0480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:43.566982 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a15cfc-cc80-4d14-ab0e-f870962a0480 admin admin] PATH_INFO: `/v3/users/fdca1b4d86984198a9c79c40098b5fa8` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:43.571407 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a15cfc-cc80-4d14-ab0e-f870962a0480 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdca1b4d86984198a9c79c40098b5fa8)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:43.576690 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a15cfc-cc80-4d14-ab0e-f870962a0480 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:43.677477 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-45a15cfc-cc80-4d14-ab0e-f870962a0480 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdca1b4d86984198a9c79c40098b5fa8'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:43.692643 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-45a15cfc-cc80-4d14-ab0e-f870962a0480 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdca1b4d86984198a9c79c40098b5fa8'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:43.715799 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-45a15cfc-cc80-4d14-ab0e-f870962a0480 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdca1b4d86984198a9c79c40098b5fa8'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:43.716805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-45a15cfc-cc80-4d14-ab0e-f870962a0480 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdca1b4d86984198a9c79c40098b5fa8'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:43.728938 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-45a15cfc-cc80-4d14-ab0e-f870962a0480 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdca1b4d86984198a9c79c40098b5fa8'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:43.741747 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1036/2087] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:04:43 2022] DELETE /identity/v3/users/fdca1b4d86984198a9c79c40098b5fa8 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:04:43.747811 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d70b09-6938-43f4-95b9-f456f18bd6eb None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:43.748618 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d70b09-6938-43f4-95b9-f456f18bd6eb None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:43.748981 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d70b09-6938-43f4-95b9-f456f18bd6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:43.788826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-64d70b09-6938-43f4-95b9-f456f18bd6eb None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:04:43.790826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-64d70b09-6938-43f4-95b9-f456f18bd6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:43.809573 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a15cfc-cc80-4d14-ab0e-f870962a0480 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:43.811726 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-45a15cfc-cc80-4d14-ab0e-f870962a0480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:43.925087 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1052/2088] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:04:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:04:43.937909 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d70b09-6938-43f4-95b9-f456f18bd6eb None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:43.939337 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-64d70b09-6938-43f4-95b9-f456f18bd6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:43.975523 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a15cfc-cc80-4d14-ab0e-f870962a0480 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:43.991583 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c6e29-3b48-483f-8b10-026fc1776bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:43.993238 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c6e29-3b48-483f-8b10-026fc1776bd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:43.993238 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c6e29-3b48-483f-8b10-026fc1776bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:43.993238 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c6e29-3b48-483f-8b10-026fc1776bd3 admin admin] PATH_INFO: `/v3/users/c1c66c943ea14a3c852843c598559144` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:44.016652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c6e29-3b48-483f-8b10-026fc1776bd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1c66c943ea14a3c852843c598559144)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:44.020269 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c6e29-3b48-483f-8b10-026fc1776bd3 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:44.086922 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d70b09-6938-43f4-95b9-f456f18bd6eb None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:44.099370 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5661e583-84ef-45c0-9c48-bafd49cdd8d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:44.100484 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661e583-84ef-45c0-9c48-bafd49cdd8d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:44.101506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661e583-84ef-45c0-9c48-bafd49cdd8d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:44.101932 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661e583-84ef-45c0-9c48-bafd49cdd8d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:44.117066 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:44.136744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5661e583-84ef-45c0-9c48-bafd49cdd8d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:44.136744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5661e583-84ef-45c0-9c48-bafd49cdd8d0 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:44.144993 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-fc2c6e29-3b48-483f-8b10-026fc1776bd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1c66c943ea14a3c852843c598559144'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:44.154931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-fc2c6e29-3b48-483f-8b10-026fc1776bd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1c66c943ea14a3c852843c598559144'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:44.155478 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1053/2089] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:04:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:04:44.185163 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-fc2c6e29-3b48-483f-8b10-026fc1776bd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1c66c943ea14a3c852843c598559144'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:44.196211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-fc2c6e29-3b48-483f-8b10-026fc1776bd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1c66c943ea14a3c852843c598559144'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:44.205678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-fc2c6e29-3b48-483f-8b10-026fc1776bd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1c66c943ea14a3c852843c598559144'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:44.212513 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1037/2090] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:04:43 2022] DELETE /identity/v3/users/c1c66c943ea14a3c852843c598559144 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:04:44.217293 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c078ba5-5491-4ad9-8542-ee1367353316 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:44.217785 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c078ba5-5491-4ad9-8542-ee1367353316 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:44.218817 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c078ba5-5491-4ad9-8542-ee1367353316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:44.250239 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-8c078ba5-5491-4ad9-8542-ee1367353316 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:04:44.252202 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-8c078ba5-5491-4ad9-8542-ee1367353316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:44.442102 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1054/2091] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:04:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:04:44.478249 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c6e29-3b48-483f-8b10-026fc1776bd3 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:44.484666 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-fc2c6e29-3b48-483f-8b10-026fc1776bd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:44.720063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c6e29-3b48-483f-8b10-026fc1776bd3 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:44.740847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84bc17-fc23-4439-b86f-38a9cd14ac6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:44.741867 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84bc17-fc23-4439-b86f-38a9cd14ac6e service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:44.742329 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84bc17-fc23-4439-b86f-38a9cd14ac6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:44.742793 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84bc17-fc23-4439-b86f-38a9cd14ac6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:44.767539 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c078ba5-5491-4ad9-8542-ee1367353316 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:44.769940 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-8c078ba5-5491-4ad9-8542-ee1367353316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:44.787957 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb84bc17-fc23-4439-b86f-38a9cd14ac6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:44.796454 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb84bc17-fc23-4439-b86f-38a9cd14ac6e service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:44.824865 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c078ba5-5491-4ad9-8542-ee1367353316 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:44.868987 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be94588-9c7a-437b-8540-097fb421b7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:44.875307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be94588-9c7a-437b-8540-097fb421b7d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:44.876865 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be94588-9c7a-437b-8540-097fb421b7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:44.879042 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be94588-9c7a-437b-8540-097fb421b7d6 admin admin] PATH_INFO: `/v3/projects/37ce1647a63d44028a709829c6e91fed` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:44.886289 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1038/2092] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:04:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:04:44.896327 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be94588-9c7a-437b-8540-097fb421b7d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37ce1647a63d44028a709829c6e91fed)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:44.902856 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be94588-9c7a-437b-8540-097fb421b7d6 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:44.912482 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:45.021202 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-2be94588-9c7a-437b-8540-097fb421b7d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37ce1647a63d44028a709829c6e91fed', 'request_id': 'req-2be94588-9c7a-437b-8540-097fb421b7d6'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:45.041360 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-2be94588-9c7a-437b-8540-097fb421b7d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37ce1647a63d44028a709829c6e91fed', 'request_id': 'req-2be94588-9c7a-437b-8540-097fb421b7d6'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:45.072614 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-2be94588-9c7a-437b-8540-097fb421b7d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37ce1647a63d44028a709829c6e91fed', 'request_id': 'req-2be94588-9c7a-437b-8540-097fb421b7d6'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:45.090766 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-2be94588-9c7a-437b-8540-097fb421b7d6 admin admin] The token cache is being invalidate because project 37ce1647a63d44028a709829c6e91fed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:04:45.092956 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-2be94588-9c7a-437b-8540-097fb421b7d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:45.094101 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-2be94588-9c7a-437b-8540-097fb421b7d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:45.096465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1055/2093] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:04:44 2022] DELETE /identity/v3/projects/37ce1647a63d44028a709829c6e91fed => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:04:45.102131 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84bc17-fc23-4439-b86f-38a9cd14ac6e service neutron] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:45.118259 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84bc17-fc23-4439-b86f-38a9cd14ac6e service neutron] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:45.257702 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785de7f-7a4a-41c3-89fc-c1614ff17cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:45.258533 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785de7f-7a4a-41c3-89fc-c1614ff17cd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:45.258917 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785de7f-7a4a-41c3-89fc-c1614ff17cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:45.259332 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785de7f-7a4a-41c3-89fc-c1614ff17cd3 admin admin] PATH_INFO: `/v3/users/cdcf1d15ca31458e9ce1b9c4d30309d1` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:45.260958 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785de7f-7a4a-41c3-89fc-c1614ff17cd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdcf1d15ca31458e9ce1b9c4d30309d1)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:45.264124 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785de7f-7a4a-41c3-89fc-c1614ff17cd3 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:45.382256 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-1785de7f-7a4a-41c3-89fc-c1614ff17cd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdcf1d15ca31458e9ce1b9c4d30309d1'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:45.401371 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-1785de7f-7a4a-41c3-89fc-c1614ff17cd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdcf1d15ca31458e9ce1b9c4d30309d1'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:45.421638 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:45.447523 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-1785de7f-7a4a-41c3-89fc-c1614ff17cd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdcf1d15ca31458e9ce1b9c4d30309d1'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:45.448372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-1785de7f-7a4a-41c3-89fc-c1614ff17cd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdcf1d15ca31458e9ce1b9c4d30309d1'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:45.504669 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-1785de7f-7a4a-41c3-89fc-c1614ff17cd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdcf1d15ca31458e9ce1b9c4d30309d1'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:45.519733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1039/2094] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:04:45 2022] DELETE /identity/v3/users/cdcf1d15ca31458e9ce1b9c4d30309d1 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:04:45.644538 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be94588-9c7a-437b-8540-097fb421b7d6 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:45.649717 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-2be94588-9c7a-437b-8540-097fb421b7d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:45.666718 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be94588-9c7a-437b-8540-097fb421b7d6 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:45.778072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3dee2b-a6a0-48cb-8533-562631cd2b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:45.778732 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3dee2b-a6a0-48cb-8533-562631cd2b40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:45.779516 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3dee2b-a6a0-48cb-8533-562631cd2b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:45.781061 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3dee2b-a6a0-48cb-8533-562631cd2b40 admin admin] PATH_INFO: `/v3/projects/fafe2f672b7c4979bbc76dc9445bc628` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:45.784004 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3dee2b-a6a0-48cb-8533-562631cd2b40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fafe2f672b7c4979bbc76dc9445bc628)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:45.786231 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3dee2b-a6a0-48cb-8533-562631cd2b40 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:45.815045 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785de7f-7a4a-41c3-89fc-c1614ff17cd3 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:45.816550 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-1785de7f-7a4a-41c3-89fc-c1614ff17cd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:45.835209 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785de7f-7a4a-41c3-89fc-c1614ff17cd3 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:45.847609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11386d85-0aa6-46b0-8b6f-c92b7f50e85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:45.848272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11386d85-0aa6-46b0-8b6f-c92b7f50e85e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:45.848548 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11386d85-0aa6-46b0-8b6f-c92b7f50e85e admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:45.848848 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11386d85-0aa6-46b0-8b6f-c92b7f50e85e admin admin] PATH_INFO: `/v3/users/83e975aed96648fc9c06f95d49c6f6f9` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:45.850383 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11386d85-0aa6-46b0-8b6f-c92b7f50e85e admin admin] RBAC: Authorizing `identity:delete_user(user_id=83e975aed96648fc9c06f95d49c6f6f9)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:45.852362 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11386d85-0aa6-46b0-8b6f-c92b7f50e85e admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:45.868013 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-fb3dee2b-a6a0-48cb-8533-562631cd2b40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fafe2f672b7c4979bbc76dc9445bc628', 'request_id': 'req-fb3dee2b-a6a0-48cb-8533-562631cd2b40'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:45.886033 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-fb3dee2b-a6a0-48cb-8533-562631cd2b40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fafe2f672b7c4979bbc76dc9445bc628', 'request_id': 'req-fb3dee2b-a6a0-48cb-8533-562631cd2b40'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:45.906406 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-fb3dee2b-a6a0-48cb-8533-562631cd2b40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fafe2f672b7c4979bbc76dc9445bc628', 'request_id': 'req-fb3dee2b-a6a0-48cb-8533-562631cd2b40'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:45.931527 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-fb3dee2b-a6a0-48cb-8533-562631cd2b40 admin admin] The token cache is being invalidate because project fafe2f672b7c4979bbc76dc9445bc628 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:04:45.931818 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-fb3dee2b-a6a0-48cb-8533-562631cd2b40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:45.932866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-fb3dee2b-a6a0-48cb-8533-562631cd2b40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:45.939878 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1056/2095] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:04:45 2022] DELETE /identity/v3/projects/fafe2f672b7c4979bbc76dc9445bc628 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:04:45.943860 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eda502-eb83-4dcd-99e6-ae0439a9a0ce None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:45.944282 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eda502-eb83-4dcd-99e6-ae0439a9a0ce None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:45.944570 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eda502-eb83-4dcd-99e6-ae0439a9a0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:45.997669 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-11386d85-0aa6-46b0-8b6f-c92b7f50e85e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83e975aed96648fc9c06f95d49c6f6f9'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:46.002419 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-42eda502-eb83-4dcd-99e6-ae0439a9a0ce None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:04:46.003972 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-42eda502-eb83-4dcd-99e6-ae0439a9a0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:46.010836 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-11386d85-0aa6-46b0-8b6f-c92b7f50e85e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83e975aed96648fc9c06f95d49c6f6f9'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:46.036910 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:46.057483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-11386d85-0aa6-46b0-8b6f-c92b7f50e85e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83e975aed96648fc9c06f95d49c6f6f9'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:46.062719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-11386d85-0aa6-46b0-8b6f-c92b7f50e85e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83e975aed96648fc9c06f95d49c6f6f9'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:46.078412 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-11386d85-0aa6-46b0-8b6f-c92b7f50e85e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83e975aed96648fc9c06f95d49c6f6f9'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:46.087940 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1040/2096] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:04:45 2022] DELETE /identity/v3/users/83e975aed96648fc9c06f95d49c6f6f9 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:04:46.126381 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1057/2097] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:04:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:04:46.135014 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804f243-217d-4a7c-ac97-c245a20833e8 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:46.135214 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804f243-217d-4a7c-ac97-c245a20833e8 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:46.135606 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804f243-217d-4a7c-ac97-c245a20833e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:46.163190 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-2804f243-217d-4a7c-ac97-c245a20833e8 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:04:46.165031 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-2804f243-217d-4a7c-ac97-c245a20833e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:46.262705 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1041/2098] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:04:46.267243 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eda502-eb83-4dcd-99e6-ae0439a9a0ce None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:46.280171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eda502-eb83-4dcd-99e6-ae0439a9a0ce None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:46.295248 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd86c64-6f01-4f98-b52b-cece10b4166c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:46.295882 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd86c64-6f01-4f98-b52b-cece10b4166c admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:46.296167 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd86c64-6f01-4f98-b52b-cece10b4166c admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:46.296463 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd86c64-6f01-4f98-b52b-cece10b4166c admin admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:46.297231 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd86c64-6f01-4f98-b52b-cece10b4166c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:04:46.297538 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd86c64-6f01-4f98-b52b-cece10b4166c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:46.299215 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd86c64-6f01-4f98-b52b-cece10b4166c admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:46.330823 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1058/2099] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:04:46.332193 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804f243-217d-4a7c-ac97-c245a20833e8 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:46.350702 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804f243-217d-4a7c-ac97-c245a20833e8 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:46.357296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00637a-560d-42e8-93a9-9497c9d762e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:46.358487 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00637a-560d-42e8-93a9-9497c9d762e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:46.358842 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00637a-560d-42e8-93a9-9497c9d762e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:46.359197 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00637a-560d-42e8-93a9-9497c9d762e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:46.359958 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb00637a-560d-42e8-93a9-9497c9d762e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:46.362551 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb00637a-560d-42e8-93a9-9497c9d762e7 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:46.402812 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1042/2100] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:04:46 2022] POST /identity/v3/projects => generated 377 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:04:46.408060 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd86c64-6f01-4f98-b52b-cece10b4166c admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:46.426181 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00637a-560d-42e8-93a9-9497c9d762e7 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:46.429436 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-bb00637a-560d-42e8-93a9-9497c9d762e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:46.431995 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd86c64-6f01-4f98-b52b-cece10b4166c admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:46.525065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:46.535343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65802323-0b41-4c2c-a69a-162cf6a05311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:46.535973 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65802323-0b41-4c2c-a69a-162cf6a05311 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:46.536427 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65802323-0b41-4c2c-a69a-162cf6a05311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:46.536707 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65802323-0b41-4c2c-a69a-162cf6a05311 admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:46.537390 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65802323-0b41-4c2c-a69a-162cf6a05311 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:46.539320 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65802323-0b41-4c2c-a69a-162cf6a05311 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:46.561084 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00637a-560d-42e8-93a9-9497c9d762e7 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:46.561535 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1059/2101] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:04:46 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:04:46.562364 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65802323-0b41-4c2c-a69a-162cf6a05311 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:46.564112 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-65802323-0b41-4c2c-a69a-162cf6a05311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:46.568839 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b599e2-9bd2-41b9-9a2a-99a2fc65ee94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:46.569484 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b599e2-9bd2-41b9-9a2a-99a2fc65ee94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:46.569767 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b599e2-9bd2-41b9-9a2a-99a2fc65ee94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:46.570043 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b599e2-9bd2-41b9-9a2a-99a2fc65ee94 admin admin] PATH_INFO: `/v3/projects/3e6d064e6ebd4cef94edc8afed451f38` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:46.571272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b599e2-9bd2-41b9-9a2a-99a2fc65ee94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e6d064e6ebd4cef94edc8afed451f38)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:46.573245 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b599e2-9bd2-41b9-9a2a-99a2fc65ee94 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:46.578580 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65802323-0b41-4c2c-a69a-162cf6a05311 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:46.584311 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02e1ee-6109-45eb-8e03-cfe33a99d01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:46.585060 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02e1ee-6109-45eb-8e03-cfe33a99d01d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:46.585352 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02e1ee-6109-45eb-8e03-cfe33a99d01d admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:46.585620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02e1ee-6109-45eb-8e03-cfe33a99d01d admin admin] PATH_INFO: `/v3/projects/2bb000ac20014d618784b262288bac32` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:46.586775 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e02e1ee-6109-45eb-8e03-cfe33a99d01d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bb000ac20014d618784b262288bac32)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:46.588702 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e02e1ee-6109-45eb-8e03-cfe33a99d01d admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:46.629088 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-e5b599e2-9bd2-41b9-9a2a-99a2fc65ee94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e6d064e6ebd4cef94edc8afed451f38', 'request_id': 'req-e5b599e2-9bd2-41b9-9a2a-99a2fc65ee94'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:46.638416 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-e5b599e2-9bd2-41b9-9a2a-99a2fc65ee94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e6d064e6ebd4cef94edc8afed451f38', 'request_id': 'req-e5b599e2-9bd2-41b9-9a2a-99a2fc65ee94'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:46.645826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-e5b599e2-9bd2-41b9-9a2a-99a2fc65ee94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e6d064e6ebd4cef94edc8afed451f38', 'request_id': 'req-e5b599e2-9bd2-41b9-9a2a-99a2fc65ee94'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:46.657208 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-e5b599e2-9bd2-41b9-9a2a-99a2fc65ee94 admin admin] The token cache is being invalidate because project 3e6d064e6ebd4cef94edc8afed451f38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:04:46.657490 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-e5b599e2-9bd2-41b9-9a2a-99a2fc65ee94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:46.658151 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-e5b599e2-9bd2-41b9-9a2a-99a2fc65ee94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:46.659604 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-9e02e1ee-6109-45eb-8e03-cfe33a99d01d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bb000ac20014d618784b262288bac32', 'request_id': 'req-9e02e1ee-6109-45eb-8e03-cfe33a99d01d'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:46.659936 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1043/2102] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:04:46 2022] DELETE /identity/v3/projects/3e6d064e6ebd4cef94edc8afed451f38 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:04:46.660802 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b599e2-9bd2-41b9-9a2a-99a2fc65ee94 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:46.662603 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-e5b599e2-9bd2-41b9-9a2a-99a2fc65ee94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:46.668287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-9e02e1ee-6109-45eb-8e03-cfe33a99d01d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bb000ac20014d618784b262288bac32', 'request_id': 'req-9e02e1ee-6109-45eb-8e03-cfe33a99d01d'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:46.680528 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-9e02e1ee-6109-45eb-8e03-cfe33a99d01d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bb000ac20014d618784b262288bac32', 'request_id': 'req-9e02e1ee-6109-45eb-8e03-cfe33a99d01d'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:46.693526 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-9e02e1ee-6109-45eb-8e03-cfe33a99d01d admin admin] The token cache is being invalidate because project 2bb000ac20014d618784b262288bac32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:04:46.693826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-9e02e1ee-6109-45eb-8e03-cfe33a99d01d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:46.694492 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-9e02e1ee-6109-45eb-8e03-cfe33a99d01d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:46.696277 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1060/2103] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:04:46 2022] DELETE /identity/v3/projects/2bb000ac20014d618784b262288bac32 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:04:46.697260 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c11441-51f6-4bbf-b2cb-a68c61e28e05 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:46.697557 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c11441-51f6-4bbf-b2cb-a68c61e28e05 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:46.697825 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c11441-51f6-4bbf-b2cb-a68c61e28e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:46.720330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-96c11441-51f6-4bbf-b2cb-a68c61e28e05 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:04:46.721576 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-96c11441-51f6-4bbf-b2cb-a68c61e28e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:46.737697 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:46.744497 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b599e2-9bd2-41b9-9a2a-99a2fc65ee94 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:46.800306 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1061/2104] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:04:46.802270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b1ac4-d0ef-4b5c-8383-7acc504c8052 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:46.802728 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b1ac4-d0ef-4b5c-8383-7acc504c8052 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:46.803139 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b1ac4-d0ef-4b5c-8383-7acc504c8052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:46.832351 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a90906b-e8b6-4b52-946a-0f3ffc59e1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:46.833047 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90906b-e8b6-4b52-946a-0f3ffc59e1ff admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:46.833364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90906b-e8b6-4b52-946a-0f3ffc59e1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:46.833608 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90906b-e8b6-4b52-946a-0f3ffc59e1ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:46.834310 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a90906b-e8b6-4b52-946a-0f3ffc59e1ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:46.836030 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a90906b-e8b6-4b52-946a-0f3ffc59e1ff admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:46.850520 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1044/2105] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:04:46.850520 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132b53c-1d65-483e-9c1b-1c5b47e630de None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:46.850520 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132b53c-1d65-483e-9c1b-1c5b47e630de None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:46.850520 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132b53c-1d65-483e-9c1b-1c5b47e630de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:46.858241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:46.867670 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-f83b1ac4-d0ef-4b5c-8383-7acc504c8052 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:04:46.869215 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-f83b1ac4-d0ef-4b5c-8383-7acc504c8052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:46.871745 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-d132b53c-1d65-483e-9c1b-1c5b47e630de None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:04:46.874580 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-d132b53c-1d65-483e-9c1b-1c5b47e630de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:46.884616 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1062/2106] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:04:46.885525 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b1ac4-d0ef-4b5c-8383-7acc504c8052 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:46.888924 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1045/2107] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:04:46.890063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132b53c-1d65-483e-9c1b-1c5b47e630de None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:46.894335 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-f83b1ac4-d0ef-4b5c-8383-7acc504c8052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:46.907502 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132b53c-1d65-483e-9c1b-1c5b47e630de None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:46.908319 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b1ac4-d0ef-4b5c-8383-7acc504c8052 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:46.913792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa5988-0733-4f4d-8815-10af31ecae8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:46.914084 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196e8f8-99cb-4a96-939d-836de46f1f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:46.914718 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196e8f8-99cb-4a96-939d-836de46f1f32 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:46.915001 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196e8f8-99cb-4a96-939d-836de46f1f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:46.915270 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196e8f8-99cb-4a96-939d-836de46f1f32 admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:46.915570 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa5988-0733-4f4d-8815-10af31ecae8f admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:46.915978 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196e8f8-99cb-4a96-939d-836de46f1f32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:04:46.916283 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196e8f8-99cb-4a96-939d-836de46f1f32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:46.916394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa5988-0733-4f4d-8815-10af31ecae8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:46.916880 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa5988-0733-4f4d-8815-10af31ecae8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:46.917710 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa5988-0733-4f4d-8815-10af31ecae8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:46.917961 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196e8f8-99cb-4a96-939d-836de46f1f32 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:46.920157 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa5988-0733-4f4d-8815-10af31ecae8f admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:46.932678 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1063/2108] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:04:46.933174 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742ab76-cf35-4a0b-b8d9-555404aa59ef None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:46.933505 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742ab76-cf35-4a0b-b8d9-555404aa59ef None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:46.933784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742ab76-cf35-4a0b-b8d9-555404aa59ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:46.945013 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1046/2109] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:04:46.945773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196e8f8-99cb-4a96-939d-836de46f1f32 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:46.956691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-4742ab76-cf35-4a0b-b8d9-555404aa59ef None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:04:46.957896 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-4742ab76-cf35-4a0b-b8d9-555404aa59ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:46.959392 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196e8f8-99cb-4a96-939d-836de46f1f32 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:46.964017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9375bf-d2fd-4be6-98a6-f9c99050ea24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:46.964784 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9375bf-d2fd-4be6-98a6-f9c99050ea24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:46.965061 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9375bf-d2fd-4be6-98a6-f9c99050ea24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:46.965334 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9375bf-d2fd-4be6-98a6-f9c99050ea24 admin admin] PATH_INFO: `/v3/projects/6a68616269204bec9023216f4d33c7b7/users/aae3fd65389f4ec8933b0508ba677e50/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:46.967327 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1064/2110] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:04:46.968285 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742ab76-cf35-4a0b-b8d9-555404aa59ef None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:46.980151 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742ab76-cf35-4a0b-b8d9-555404aa59ef None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:46.992521 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f9183-8bbc-4843-8192-1f5db2f50c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:46.994154 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f9183-8bbc-4843-8192-1f5db2f50c0d admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:46.994717 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f9183-8bbc-4843-8192-1f5db2f50c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:46.995109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f9183-8bbc-4843-8192-1f5db2f50c0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:46.995790 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f9183-8bbc-4843-8192-1f5db2f50c0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:46.995892 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9375bf-d2fd-4be6-98a6-f9c99050ea24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a68616269204bec9023216f4d33c7b7, user_id=aae3fd65389f4ec8933b0508ba677e50, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:46.999031 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f9183-8bbc-4843-8192-1f5db2f50c0d admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:46.999173 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9375bf-d2fd-4be6-98a6-f9c99050ea24 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:47.021181 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1047/2111] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:04:46 2022] PUT /identity/v3/projects/6a68616269204bec9023216f4d33c7b7/users/aae3fd65389f4ec8933b0508ba677e50/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:04:47.022298 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9375bf-d2fd-4be6-98a6-f9c99050ea24 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:47.039445 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1065/2112] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:04:46 2022] POST /identity/v3/projects => generated 365 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:04:47.041450 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc8ff6-efb6-421f-8d8c-0ca33eefb6c2 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:47.041847 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc8ff6-efb6-421f-8d8c-0ca33eefb6c2 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:47.043267 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc8ff6-efb6-421f-8d8c-0ca33eefb6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:47.089422 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-56dc8ff6-efb6-421f-8d8c-0ca33eefb6c2 None None] MFA Rules not processed for user `aae3fd65389f4ec8933b0508ba677e50`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:04:47.090665 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-56dc8ff6-efb6-421f-8d8c-0ca33eefb6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:47.109795 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9375bf-d2fd-4be6-98a6-f9c99050ea24 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:47.187917 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1066/2113] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:04:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:04:47.188649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc8ff6-efb6-421f-8d8c-0ca33eefb6c2 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:47.197578 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac5aa5-2048-4242-8e98-f4751b4f34b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:47.198344 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac5aa5-2048-4242-8e98-f4751b4f34b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:47.198633 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac5aa5-2048-4242-8e98-f4751b4f34b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:47.198910 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac5aa5-2048-4242-8e98-f4751b4f34b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:47.199613 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac5aa5-2048-4242-8e98-f4751b4f34b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:04:47.199808 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc8ff6-efb6-421f-8d8c-0ca33eefb6c2 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:47.199977 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac5aa5-2048-4242-8e98-f4751b4f34b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:47.201897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac5aa5-2048-4242-8e98-f4751b4f34b2 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:47.207240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec42fdaa-695e-4726-85f5-eca1021c8ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:47.207863 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42fdaa-695e-4726-85f5-eca1021c8ad2 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:47.208148 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42fdaa-695e-4726-85f5-eca1021c8ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:47.208421 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42fdaa-695e-4726-85f5-eca1021c8ad2 admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:47.209123 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec42fdaa-695e-4726-85f5-eca1021c8ad2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:47.212005 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec42fdaa-695e-4726-85f5-eca1021c8ad2 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:47.219035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1048/2114] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:04:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:04:47.228539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac5aa5-2048-4242-8e98-f4751b4f34b2 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:47.245054 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac5aa5-2048-4242-8e98-f4751b4f34b2 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:47.245771 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1067/2115] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:04:47 2022] POST /identity/v3/users => generated 283 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:04:47.246582 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec42fdaa-695e-4726-85f5-eca1021c8ad2 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:47.253734 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90bb27a-2dfd-4d98-ab71-9dbb286bdd26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:47.254360 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90bb27a-2dfd-4d98-ab71-9dbb286bdd26 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:47.254644 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90bb27a-2dfd-4d98-ab71-9dbb286bdd26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:47.254919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90bb27a-2dfd-4d98-ab71-9dbb286bdd26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:47.255597 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90bb27a-2dfd-4d98-ab71-9dbb286bdd26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:47.257535 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90bb27a-2dfd-4d98-ab71-9dbb286bdd26 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:47.258201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec42fdaa-695e-4726-85f5-eca1021c8ad2 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:47.264241 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d183e-06aa-4f89-bfb7-57b7cc0029c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:47.265182 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d183e-06aa-4f89-bfb7-57b7cc0029c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:47.265671 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d183e-06aa-4f89-bfb7-57b7cc0029c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:47.266127 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d183e-06aa-4f89-bfb7-57b7cc0029c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:47.267030 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d183e-06aa-4f89-bfb7-57b7cc0029c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:47.269426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d183e-06aa-4f89-bfb7-57b7cc0029c5 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:47.292542 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1049/2116] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:04:47 2022] POST /identity/v3/projects => generated 375 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:04:47.293790 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90bb27a-2dfd-4d98-ab71-9dbb286bdd26 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:47.308448 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1068/2117] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:04:47 2022] POST /identity/v3/projects => generated 389 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:04:47.308448 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d183e-06aa-4f89-bfb7-57b7cc0029c5 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:47.345083 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:47.397027 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d183e-06aa-4f89-bfb7-57b7cc0029c5 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:47.401304 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:47.405004 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5d3a5-809e-44ef-8011-06ac73de9ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:47.405828 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5d3a5-809e-44ef-8011-06ac73de9ab6 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:47.406228 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5d3a5-809e-44ef-8011-06ac73de9ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:47.406630 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5d3a5-809e-44ef-8011-06ac73de9ab6 admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:47.407529 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a5d3a5-809e-44ef-8011-06ac73de9ab6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:47.409622 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a5d3a5-809e-44ef-8011-06ac73de9ab6 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:47.437543 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1069/2118] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:04:47 2022] POST /identity/v3/users => generated 288 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:04:47.438531 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5d3a5-809e-44ef-8011-06ac73de9ab6 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:47.445438 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90bb27a-2dfd-4d98-ab71-9dbb286bdd26 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:47.448914 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5d3a5-809e-44ef-8011-06ac73de9ab6 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:47.450407 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539e88f-696d-44bc-99cc-2d1aebc8de99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:47.451427 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539e88f-696d-44bc-99cc-2d1aebc8de99 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:47.451690 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539e88f-696d-44bc-99cc-2d1aebc8de99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:47.451965 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539e88f-696d-44bc-99cc-2d1aebc8de99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:47.452708 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2539e88f-696d-44bc-99cc-2d1aebc8de99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:47.454301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2539e88f-696d-44bc-99cc-2d1aebc8de99 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:47.454833 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee8f323-7845-4579-8e60-aeeae74c34ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:47.455542 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8f323-7845-4579-8e60-aeeae74c34ba admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:47.455890 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8f323-7845-4579-8e60-aeeae74c34ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:47.456238 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8f323-7845-4579-8e60-aeeae74c34ba admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:47.457095 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee8f323-7845-4579-8e60-aeeae74c34ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:47.459120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee8f323-7845-4579-8e60-aeeae74c34ba admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:47.469148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1050/2119] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:04:47.469148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539e88f-696d-44bc-99cc-2d1aebc8de99 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:47.479874 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539e88f-696d-44bc-99cc-2d1aebc8de99 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:47.486865 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b526edd-c2a3-495c-9df1-be399e046ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:47.487573 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b526edd-c2a3-495c-9df1-be399e046ddd admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:47.487931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b526edd-c2a3-495c-9df1-be399e046ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:47.488276 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b526edd-c2a3-495c-9df1-be399e046ddd admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:47.489192 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b526edd-c2a3-495c-9df1-be399e046ddd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:47.493085 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1070/2120] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:04:47 2022] POST /identity/v3/users => generated 295 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:04:47.493235 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b526edd-c2a3-495c-9df1-be399e046ddd admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:47.493323 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee8f323-7845-4579-8e60-aeeae74c34ba admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:47.505700 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1051/2121] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:04:47.505862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee8f323-7845-4579-8e60-aeeae74c34ba admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:47.506040 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b526edd-c2a3-495c-9df1-be399e046ddd admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:47.510214 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6074a-1c2b-4a36-b815-cea3af356130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:47.511469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6074a-1c2b-4a36-b815-cea3af356130 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:47.511812 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6074a-1c2b-4a36-b815-cea3af356130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:47.512116 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6074a-1c2b-4a36-b815-cea3af356130 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:47.514966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f6074a-1c2b-4a36-b815-cea3af356130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:47.515836 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f6074a-1c2b-4a36-b815-cea3af356130 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:47.522990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b526edd-c2a3-495c-9df1-be399e046ddd admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:47.531357 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1071/2122] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:04:47.532085 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbad626-e162-483a-ad84-013160aa4305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:47.533331 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbad626-e162-483a-ad84-013160aa4305 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:47.533836 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbad626-e162-483a-ad84-013160aa4305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:47.534118 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbad626-e162-483a-ad84-013160aa4305 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:47.534933 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbad626-e162-483a-ad84-013160aa4305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:47.536527 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbad626-e162-483a-ad84-013160aa4305 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:47.540221 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6074a-1c2b-4a36-b815-cea3af356130 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:47.548639 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1052/2123] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:04:47.549440 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbad626-e162-483a-ad84-013160aa4305 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:47.555220 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6074a-1c2b-4a36-b815-cea3af356130 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:47.559597 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbad626-e162-483a-ad84-013160aa4305 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:47.561373 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c96ce-ba9a-4553-98d2-6f05fa27820a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:47.562401 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c96ce-ba9a-4553-98d2-6f05fa27820a admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:47.562697 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c96ce-ba9a-4553-98d2-6f05fa27820a admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:47.562975 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c96ce-ba9a-4553-98d2-6f05fa27820a admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:47.563671 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c96ce-ba9a-4553-98d2-6f05fa27820a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:47.565189 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c6e77-16f8-4de4-869d-7c24c34f0ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:47.565334 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c96ce-ba9a-4553-98d2-6f05fa27820a admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:47.565922 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c6e77-16f8-4de4-869d-7c24c34f0ba6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:47.566275 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c6e77-16f8-4de4-869d-7c24c34f0ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:47.566626 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c6e77-16f8-4de4-869d-7c24c34f0ba6 admin admin] PATH_INFO: `/v3/projects/0968f83baf04490aaed1cddc81fc0d11/users/868b52d1f3494ece9fee62a577df6bc2/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:47.577796 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1072/2124] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:04:47.582643 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c96ce-ba9a-4553-98d2-6f05fa27820a admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:47.583899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c6e77-16f8-4de4-869d-7c24c34f0ba6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0968f83baf04490aaed1cddc81fc0d11, user_id=868b52d1f3494ece9fee62a577df6bc2, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:47.585045 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-ee0c96ce-ba9a-4553-98d2-6f05fa27820a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:47.586755 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c6e77-16f8-4de4-869d-7c24c34f0ba6 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:47.603338 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c96ce-ba9a-4553-98d2-6f05fa27820a admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:47.610609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1053/2125] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:04:47 2022] PUT /identity/v3/projects/0968f83baf04490aaed1cddc81fc0d11/users/868b52d1f3494ece9fee62a577df6bc2/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:04:47.610609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c6e77-16f8-4de4-869d-7c24c34f0ba6 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:47.680643 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa3fcc-6918-4d78-bcad-214f7820ca7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:47.681295 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa3fcc-6918-4d78-bcad-214f7820ca7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:47.681573 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa3fcc-6918-4d78-bcad-214f7820ca7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:47.681847 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa3fcc-6918-4d78-bcad-214f7820ca7e admin admin] PATH_INFO: `/v3/users/4e575ea70ead4e4dbae3a1589b3898be` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:47.683014 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa3fcc-6918-4d78-bcad-214f7820ca7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e575ea70ead4e4dbae3a1589b3898be)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:47.683796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c6e77-16f8-4de4-869d-7c24c34f0ba6 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:47.686660 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa3fcc-6918-4d78-bcad-214f7820ca7e admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:47.690538 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b7cc6-54cb-44bc-8aca-b29f4886053f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:47.691115 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b7cc6-54cb-44bc-8aca-b29f4886053f admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:47.691401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b7cc6-54cb-44bc-8aca-b29f4886053f admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:47.691676 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b7cc6-54cb-44bc-8aca-b29f4886053f admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:47.692432 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6b7cc6-54cb-44bc-8aca-b29f4886053f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:47.694295 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6b7cc6-54cb-44bc-8aca-b29f4886053f admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:47.709354 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1054/2126] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:04:47.710727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b7cc6-54cb-44bc-8aca-b29f4886053f admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:47.724002 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b7cc6-54cb-44bc-8aca-b29f4886053f admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:47.731125 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c897b164-d7b7-4125-ba56-0bc6c35192a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:47.733024 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897b164-d7b7-4125-ba56-0bc6c35192a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:47.733360 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897b164-d7b7-4125-ba56-0bc6c35192a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:47.733698 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897b164-d7b7-4125-ba56-0bc6c35192a4 admin admin] PATH_INFO: `/v3/projects/fb987bbda16344edb1c01b700e112ac6/users/031a09c7f7f24681a44c8025af34f75e/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:47.757231 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c897b164-d7b7-4125-ba56-0bc6c35192a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb987bbda16344edb1c01b700e112ac6, user_id=031a09c7f7f24681a44c8025af34f75e, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:47.760266 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c897b164-d7b7-4125-ba56-0bc6c35192a4 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:47.767592 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ebaa3fcc-6918-4d78-bcad-214f7820ca7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e575ea70ead4e4dbae3a1589b3898be'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:47.778251 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1055/2127] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:04:47 2022] PUT /identity/v3/projects/fb987bbda16344edb1c01b700e112ac6/users/031a09c7f7f24681a44c8025af34f75e/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:04:47.779696 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76209d62-6e95-4f63-af33-d32a8c89fbf0 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:47.780061 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76209d62-6e95-4f63-af33-d32a8c89fbf0 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:47.780430 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76209d62-6e95-4f63-af33-d32a8c89fbf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:47.783814 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ebaa3fcc-6918-4d78-bcad-214f7820ca7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e575ea70ead4e4dbae3a1589b3898be'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:47.793005 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ebaa3fcc-6918-4d78-bcad-214f7820ca7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e575ea70ead4e4dbae3a1589b3898be'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:47.806811 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=84,keystone:INSERT=7,keystone:DELETE=9 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:47.810725 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ebaa3fcc-6918-4d78-bcad-214f7820ca7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e575ea70ead4e4dbae3a1589b3898be'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:47.811842 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ebaa3fcc-6918-4d78-bcad-214f7820ca7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e575ea70ead4e4dbae3a1589b3898be'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:47.813416 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-76209d62-6e95-4f63-af33-d32a8c89fbf0 None None] MFA Rules not processed for user `868b52d1f3494ece9fee62a577df6bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:04:47.814633 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-76209d62-6e95-4f63-af33-d32a8c89fbf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:47.820675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1073/2128] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:04:47 2022] DELETE /identity/v3/users/4e575ea70ead4e4dbae3a1589b3898be => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:04:47.821590 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa3fcc-6918-4d78-bcad-214f7820ca7e admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:47.823032 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-ebaa3fcc-6918-4d78-bcad-214f7820ca7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:47.863591 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:47.903394 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1056/2129] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:04:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:04:47.903731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa3fcc-6918-4d78-bcad-214f7820ca7e admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:47.905019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd07fb5-1895-47f3-a73f-83a5c2c6b06d None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:47.905323 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd07fb5-1895-47f3-a73f-83a5c2c6b06d None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:47.905596 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd07fb5-1895-47f3-a73f-83a5c2c6b06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:47.915372 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36900eda-8b5f-4930-8930-a8853482b655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:47.916478 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36900eda-8b5f-4930-8930-a8853482b655 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:47.917014 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36900eda-8b5f-4930-8930-a8853482b655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:47.917364 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36900eda-8b5f-4930-8930-a8853482b655 admin admin] PATH_INFO: `/v3/projects/aa4789b47e0f47b0acd1cc4919593c33/users/160c88a7ba0144d4b1bc953767b12bb7/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:47.942912 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-9dd07fb5-1895-47f3-a73f-83a5c2c6b06d None None] MFA Rules not processed for user `031a09c7f7f24681a44c8025af34f75e`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:04:47.944139 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-9dd07fb5-1895-47f3-a73f-83a5c2c6b06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:47.944291 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36900eda-8b5f-4930-8930-a8853482b655 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa4789b47e0f47b0acd1cc4919593c33, user_id=160c88a7ba0144d4b1bc953767b12bb7, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:47.947119 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36900eda-8b5f-4930-8930-a8853482b655 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:47.963709 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1074/2130] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:04:47 2022] PUT /identity/v3/projects/aa4789b47e0f47b0acd1cc4919593c33/users/160c88a7ba0144d4b1bc953767b12bb7/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:04:47.966094 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8756c9-482d-48e9-a8d7-f233e4140353 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:47.966670 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8756c9-482d-48e9-a8d7-f233e4140353 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:47.967101 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8756c9-482d-48e9-a8d7-f233e4140353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:47.987502 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-7f8756c9-482d-48e9-a8d7-f233e4140353 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:04:47.988755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-7f8756c9-482d-48e9-a8d7-f233e4140353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:48.024172 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1057/2131] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:04:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:04:48.025870 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd07fb5-1895-47f3-a73f-83a5c2c6b06d None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:48.028964 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-9dd07fb5-1895-47f3-a73f-83a5c2c6b06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:48.044536 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd07fb5-1895-47f3-a73f-83a5c2c6b06d None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:48.052570 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1075/2132] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:04:48.054055 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb64908-0b48-48d9-a281-bd6d342a365c None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:48.054357 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb64908-0b48-48d9-a281-bd6d342a365c None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:48.054622 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb64908-0b48-48d9-a281-bd6d342a365c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:48.076164 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-edb64908-0b48-48d9-a281-bd6d342a365c None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:04:48.077773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-edb64908-0b48-48d9-a281-bd6d342a365c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:48.089674 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1076/2133] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:04:48.089674 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb64908-0b48-48d9-a281-bd6d342a365c None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:48.089674 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-edb64908-0b48-48d9-a281-bd6d342a365c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:48.108874 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb64908-0b48-48d9-a281-bd6d342a365c None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:48.116106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a3718-d647-44f1-953b-27a4d80cc822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:48.116787 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a3718-d647-44f1-953b-27a4d80cc822 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:48.117070 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a3718-d647-44f1-953b-27a4d80cc822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:48.117351 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a3718-d647-44f1-953b-27a4d80cc822 admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:48.118028 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a3718-d647-44f1-953b-27a4d80cc822 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:48.119826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486026c-6c60-400b-a106-c77ff96f6d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:48.120003 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a3718-d647-44f1-953b-27a4d80cc822 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:48.121023 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486026c-6c60-400b-a106-c77ff96f6d7f admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:48.121319 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486026c-6c60-400b-a106-c77ff96f6d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:48.121598 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486026c-6c60-400b-a106-c77ff96f6d7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:48.122430 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d486026c-6c60-400b-a106-c77ff96f6d7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:48.125330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d486026c-6c60-400b-a106-c77ff96f6d7f admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:48.161863 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1058/2134] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:04:48 2022] POST /identity/v3/projects => generated 365 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:04:48.162588 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a3718-d647-44f1-953b-27a4d80cc822 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:48.167050 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-502a3718-d647-44f1-953b-27a4d80cc822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:48.167201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1077/2135] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:04:48 2022] POST /identity/v3/projects => generated 375 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:04:48.167201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486026c-6c60-400b-a106-c77ff96f6d7f admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:48.168845 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-d486026c-6c60-400b-a106-c77ff96f6d7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:48.226876 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a3718-d647-44f1-953b-27a4d80cc822 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:48.230906 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24ff99-5b8f-406e-90f9-46025ce05ffb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:48.231537 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24ff99-5b8f-406e-90f9-46025ce05ffb service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:48.231865 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24ff99-5b8f-406e-90f9-46025ce05ffb service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:48.232364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24ff99-5b8f-406e-90f9-46025ce05ffb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:48.233863 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:48.238672 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486026c-6c60-400b-a106-c77ff96f6d7f admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:48.244873 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb6302-d819-4027-85a8-88a514510fc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:48.245702 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb6302-d819-4027-85a8-88a514510fc1 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:48.246642 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb6302-d819-4027-85a8-88a514510fc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:48.246942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb6302-d819-4027-85a8-88a514510fc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:48.250832 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:48.304809 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a24ff99-5b8f-406e-90f9-46025ce05ffb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:48.306255 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a24ff99-5b8f-406e-90f9-46025ce05ffb service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:48.318558 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb6302-d819-4027-85a8-88a514510fc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:48.319099 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1059/2136] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:04:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:04:48.319772 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24ff99-5b8f-406e-90f9-46025ce05ffb service neutron] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:48.320001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb6302-d819-4027-85a8-88a514510fc1 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:48.330198 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24ff99-5b8f-406e-90f9-46025ce05ffb service neutron] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:48.332018 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1078/2137] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:04:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:04:48.332858 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb6302-d819-4027-85a8-88a514510fc1 service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:48.335510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e824bd7-a983-4a19-af05-9880076cd4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:48.336332 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e824bd7-a983-4a19-af05-9880076cd4cb admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:48.336838 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e824bd7-a983-4a19-af05-9880076cd4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:48.337298 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e824bd7-a983-4a19-af05-9880076cd4cb admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:48.338168 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e824bd7-a983-4a19-af05-9880076cd4cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:48.340331 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e824bd7-a983-4a19-af05-9880076cd4cb admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:48.342991 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb6302-d819-4027-85a8-88a514510fc1 service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:48.348432 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714baad2-f629-4b5b-aa2d-13223681871e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:48.350351 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714baad2-f629-4b5b-aa2d-13223681871e admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:48.355227 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714baad2-f629-4b5b-aa2d-13223681871e admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:48.355227 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714baad2-f629-4b5b-aa2d-13223681871e admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:48.355227 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714baad2-f629-4b5b-aa2d-13223681871e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:48.358845 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714baad2-f629-4b5b-aa2d-13223681871e admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:48.372096 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1060/2138] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:04:48 2022] POST /identity/v3/users => generated 289 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:04:48.378173 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e824bd7-a983-4a19-af05-9880076cd4cb admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:48.392569 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1079/2139] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:04:48 2022] POST /identity/v3/users => generated 294 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:04:48.392569 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714baad2-f629-4b5b-aa2d-13223681871e admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:48.395851 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e824bd7-a983-4a19-af05-9880076cd4cb admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:48.402694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64662aa-b03c-462a-831d-bb3d34bf11d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:48.404223 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64662aa-b03c-462a-831d-bb3d34bf11d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:48.404557 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64662aa-b03c-462a-831d-bb3d34bf11d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:48.404947 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64662aa-b03c-462a-831d-bb3d34bf11d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:48.405887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64662aa-b03c-462a-831d-bb3d34bf11d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:48.406087 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714baad2-f629-4b5b-aa2d-13223681871e admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:48.408117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64662aa-b03c-462a-831d-bb3d34bf11d5 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:48.412168 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c868cc08-5d21-45d8-adf6-21aeb3c0d7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:48.412918 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868cc08-5d21-45d8-adf6-21aeb3c0d7c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:48.413316 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868cc08-5d21-45d8-adf6-21aeb3c0d7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:48.413676 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868cc08-5d21-45d8-adf6-21aeb3c0d7c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:48.414540 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c868cc08-5d21-45d8-adf6-21aeb3c0d7c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:48.417626 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c868cc08-5d21-45d8-adf6-21aeb3c0d7c4 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:48.424628 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1061/2140] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:04:48.426612 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64662aa-b03c-462a-831d-bb3d34bf11d5 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:48.442619 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64662aa-b03c-462a-831d-bb3d34bf11d5 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:48.443015 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1080/2141] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:04:48.448145 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c868cc08-5d21-45d8-adf6-21aeb3c0d7c4 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:48.450182 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa26f8-a18e-4e8e-9e51-db9c46ad6b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:48.451644 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa26f8-a18e-4e8e-9e51-db9c46ad6b33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:48.452087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa26f8-a18e-4e8e-9e51-db9c46ad6b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:48.452733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa26f8-a18e-4e8e-9e51-db9c46ad6b33 admin admin] PATH_INFO: `/v3/projects/d5d516c7152848fab0fd7d874d019db1/users/791b476baae94dc395131fd1ca5010b0/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:48.472169 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c868cc08-5d21-45d8-adf6-21aeb3c0d7c4 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:48.475176 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa26f8-a18e-4e8e-9e51-db9c46ad6b33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5d516c7152848fab0fd7d874d019db1, user_id=791b476baae94dc395131fd1ca5010b0, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:48.478141 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa26f8-a18e-4e8e-9e51-db9c46ad6b33 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:48.480801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2be31a-02d5-4b62-a4d0-45a66afc89e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:48.483040 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2be31a-02d5-4b62-a4d0-45a66afc89e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:48.483562 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2be31a-02d5-4b62-a4d0-45a66afc89e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:48.483986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2be31a-02d5-4b62-a4d0-45a66afc89e3 admin admin] PATH_INFO: `/v3/projects/cce2281b43494adfb0e707b570c30615/users/07e2532df79c49908a1ef68e37c22f98/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:48.511358 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2be31a-02d5-4b62-a4d0-45a66afc89e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cce2281b43494adfb0e707b570c30615, user_id=07e2532df79c49908a1ef68e37c22f98, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:48.514540 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2be31a-02d5-4b62-a4d0-45a66afc89e3 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:48.529860 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1062/2142] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:04:48 2022] PUT /identity/v3/projects/d5d516c7152848fab0fd7d874d019db1/users/791b476baae94dc395131fd1ca5010b0/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:04:48.531480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa26f8-a18e-4e8e-9e51-db9c46ad6b33 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:48.537153 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1081/2143] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:04:48 2022] PUT /identity/v3/projects/cce2281b43494adfb0e707b570c30615/users/07e2532df79c49908a1ef68e37c22f98/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:04:48.542384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2be31a-02d5-4b62-a4d0-45a66afc89e3 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:48.624034 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2be31a-02d5-4b62-a4d0-45a66afc89e3 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:48.624198 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa26f8-a18e-4e8e-9e51-db9c46ad6b33 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:48.628722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9bab3-929a-4231-8455-5c78bb5c1b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:48.629346 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9bab3-929a-4231-8455-5c78bb5c1b7b admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:48.629623 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9bab3-929a-4231-8455-5c78bb5c1b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:48.629894 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9bab3-929a-4231-8455-5c78bb5c1b7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:48.630675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b9bab3-929a-4231-8455-5c78bb5c1b7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:48.631634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b95f9a-4556-4b72-83fc-3c1615bcaf10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:48.632339 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b9bab3-929a-4231-8455-5c78bb5c1b7b admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:48.632870 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b95f9a-4556-4b72-83fc-3c1615bcaf10 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:48.633483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b95f9a-4556-4b72-83fc-3c1615bcaf10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:48.636722 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b95f9a-4556-4b72-83fc-3c1615bcaf10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:48.637768 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b95f9a-4556-4b72-83fc-3c1615bcaf10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:48.640828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b95f9a-4556-4b72-83fc-3c1615bcaf10 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:48.649031 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1082/2144] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:04:48.652288 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9bab3-929a-4231-8455-5c78bb5c1b7b admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:48.657055 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1063/2145] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:04:48.661723 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b95f9a-4556-4b72-83fc-3c1615bcaf10 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:48.678090 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b95f9a-4556-4b72-83fc-3c1615bcaf10 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:48.680550 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9bab3-929a-4231-8455-5c78bb5c1b7b admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:48.682781 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3819fce-60bb-4310-9a88-1bf958c9c5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:48.683448 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3819fce-60bb-4310-9a88-1bf958c9c5ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:48.683725 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3819fce-60bb-4310-9a88-1bf958c9c5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:48.683996 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3819fce-60bb-4310-9a88-1bf958c9c5ef admin admin] PATH_INFO: `/v3/projects/d5d516c7152848fab0fd7d874d019db1/users/791b476baae94dc395131fd1ca5010b0/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:48.686827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3819fce-60bb-4310-9a88-1bf958c9c5ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5d516c7152848fab0fd7d874d019db1, user_id=791b476baae94dc395131fd1ca5010b0, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:48.686973 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586422e4-1380-418a-9c62-dcd176104459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:48.687611 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586422e4-1380-418a-9c62-dcd176104459 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:48.687962 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586422e4-1380-418a-9c62-dcd176104459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:48.688440 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586422e4-1380-418a-9c62-dcd176104459 admin admin] PATH_INFO: `/v3/projects/cce2281b43494adfb0e707b570c30615/users/07e2532df79c49908a1ef68e37c22f98/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:48.689964 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3819fce-60bb-4310-9a88-1bf958c9c5ef admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:48.690610 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586422e4-1380-418a-9c62-dcd176104459 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cce2281b43494adfb0e707b570c30615, user_id=07e2532df79c49908a1ef68e37c22f98, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:48.693681 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586422e4-1380-418a-9c62-dcd176104459 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:48.704511 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1064/2146] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:04:48 2022] PUT /identity/v3/projects/d5d516c7152848fab0fd7d874d019db1/users/791b476baae94dc395131fd1ca5010b0/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:04:48.706503 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1083/2147] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:04:48 2022] PUT /identity/v3/projects/cce2281b43494adfb0e707b570c30615/users/07e2532df79c49908a1ef68e37c22f98/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:04:48.714314 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314dbc5a-3d36-4504-ad00-cb5e340e6738 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:48.714603 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314dbc5a-3d36-4504-ad00-cb5e340e6738 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:48.714937 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314dbc5a-3d36-4504-ad00-cb5e340e6738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:48.726053 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59534f-0106-44b0-931c-73a803268177 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:48.726355 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59534f-0106-44b0-931c-73a803268177 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:48.726656 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59534f-0106-44b0-931c-73a803268177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:48.766399 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-314dbc5a-3d36-4504-ad00-cb5e340e6738 None None] MFA Rules not processed for user `791b476baae94dc395131fd1ca5010b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:04:48.767857 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-314dbc5a-3d36-4504-ad00-cb5e340e6738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:48.776396 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-1c59534f-0106-44b0-931c-73a803268177 None None] MFA Rules not processed for user `07e2532df79c49908a1ef68e37c22f98`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:04:48.777837 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-1c59534f-0106-44b0-931c-73a803268177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:48.812096 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:48.813547 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:48.860132 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1065/2148] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:04:48 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:04:48.871543 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314dbc5a-3d36-4504-ad00-cb5e340e6738 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:48.873954 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-314dbc5a-3d36-4504-ad00-cb5e340e6738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:48.897212 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1084/2149] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:04:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:04:48.908913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59534f-0106-44b0-931c-73a803268177 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:48.910968 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-1c59534f-0106-44b0-931c-73a803268177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:48.941120 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314dbc5a-3d36-4504-ad00-cb5e340e6738 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:48.946116 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768edba3-4ab2-4285-9884-0f985ba1522f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:48.946740 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768edba3-4ab2-4285-9884-0f985ba1522f service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:48.947020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768edba3-4ab2-4285-9884-0f985ba1522f service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:48.947299 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768edba3-4ab2-4285-9884-0f985ba1522f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:48.958382 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768edba3-4ab2-4285-9884-0f985ba1522f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:48.959840 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768edba3-4ab2-4285-9884-0f985ba1522f service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:48.971042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1066/2150] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:04:48 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:04:48.993888 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59534f-0106-44b0-931c-73a803268177 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:49.000173 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca430b9-212a-4cb9-a3f0-1fd630de5954 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:49.000985 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca430b9-212a-4cb9-a3f0-1fd630de5954 service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:49.001317 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca430b9-212a-4cb9-a3f0-1fd630de5954 service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:49.004817 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca430b9-212a-4cb9-a3f0-1fd630de5954 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:49.007008 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-aca430b9-212a-4cb9-a3f0-1fd630de5954 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:49.063447 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768edba3-4ab2-4285-9884-0f985ba1522f service cinder] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:49.073055 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768edba3-4ab2-4285-9884-0f985ba1522f service cinder] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:49.077458 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d37c7-3d2b-4a49-999c-c4f8c6db48c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:49.078192 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d37c7-3d2b-4a49-999c-c4f8c6db48c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:49.078445 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d37c7-3d2b-4a49-999c-c4f8c6db48c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:49.078862 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d37c7-3d2b-4a49-999c-c4f8c6db48c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:49.086597 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca430b9-212a-4cb9-a3f0-1fd630de5954 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:49.088265 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca430b9-212a-4cb9-a3f0-1fd630de5954 service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:49.098287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1085/2151] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:04:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:04:49.149128 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7d37c7-3d2b-4a49-999c-c4f8c6db48c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:49.150708 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7d37c7-3d2b-4a49-999c-c4f8c6db48c1 service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:49.164372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1067/2152] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:04:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:04:49.244344 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca430b9-212a-4cb9-a3f0-1fd630de5954 service cinder] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:49.245768 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-aca430b9-212a-4cb9-a3f0-1fd630de5954 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:49.375916 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca430b9-212a-4cb9-a3f0-1fd630de5954 service cinder] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:49.383555 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d94599-b2e2-4fc4-9944-5076958e7706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:49.383555 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d94599-b2e2-4fc4-9944-5076958e7706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:49.383907 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d94599-b2e2-4fc4-9944-5076958e7706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:49.384212 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d94599-b2e2-4fc4-9944-5076958e7706 admin admin] PATH_INFO: `/v3/projects/64d5d57427914355aadd94f85db396bd` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:49.386077 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d94599-b2e2-4fc4-9944-5076958e7706 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64d5d57427914355aadd94f85db396bd)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:49.391656 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d94599-b2e2-4fc4-9944-5076958e7706 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:49.412648 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:49.473130 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-54d94599-b2e2-4fc4-9944-5076958e7706 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64d5d57427914355aadd94f85db396bd', 'request_id': 'req-54d94599-b2e2-4fc4-9944-5076958e7706'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:49.482539 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-54d94599-b2e2-4fc4-9944-5076958e7706 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64d5d57427914355aadd94f85db396bd', 'request_id': 'req-54d94599-b2e2-4fc4-9944-5076958e7706'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:49.493118 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-54d94599-b2e2-4fc4-9944-5076958e7706 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64d5d57427914355aadd94f85db396bd', 'request_id': 'req-54d94599-b2e2-4fc4-9944-5076958e7706'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:49.509695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-54d94599-b2e2-4fc4-9944-5076958e7706 admin admin] The token cache is being invalidate because project 64d5d57427914355aadd94f85db396bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:04:49.510402 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-54d94599-b2e2-4fc4-9944-5076958e7706 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:49.512489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-54d94599-b2e2-4fc4-9944-5076958e7706 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:49.524897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1086/2153] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:04:49 2022] DELETE /identity/v3/projects/64d5d57427914355aadd94f85db396bd => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:04:49.530910 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e08846-9720-4469-84c7-63262a6f7a6f None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:49.531203 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e08846-9720-4469-84c7-63262a6f7a6f None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:49.531941 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e08846-9720-4469-84c7-63262a6f7a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:49.554286 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-33e08846-9720-4469-84c7-63262a6f7a6f None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:04:49.555729 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-33e08846-9720-4469-84c7-63262a6f7a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:49.639825 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1068/2154] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:04:49.649018 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512e4a8-f337-4f2b-ab0a-c8feabbe477a None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:49.650704 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512e4a8-f337-4f2b-ab0a-c8feabbe477a None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:49.650704 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512e4a8-f337-4f2b-ab0a-c8feabbe477a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:49.707690 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-e512e4a8-f337-4f2b-ab0a-c8feabbe477a None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:04:49.708974 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e512e4a8-f337-4f2b-ab0a-c8feabbe477a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:49.720758 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1087/2155] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:04:49.722226 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e08846-9720-4469-84c7-63262a6f7a6f None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:49.732569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e08846-9720-4469-84c7-63262a6f7a6f None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:49.737626 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f0247-fac6-40e1-8155-632e1231c7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:49.738245 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f0247-fac6-40e1-8155-632e1231c7a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:49.738521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f0247-fac6-40e1-8155-632e1231c7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:49.738791 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f0247-fac6-40e1-8155-632e1231c7a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:49.739569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f0247-fac6-40e1-8155-632e1231c7a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:04:49.739908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f0247-fac6-40e1-8155-632e1231c7a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:49.743385 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f0247-fac6-40e1-8155-632e1231c7a6 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:49.750940 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:49.761748 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1069/2156] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:04:49.763564 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512e4a8-f337-4f2b-ab0a-c8feabbe477a None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:49.779953 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512e4a8-f337-4f2b-ab0a-c8feabbe477a None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:49.783003 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c384af9-65cb-4cdf-a540-9e1da1b0b1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:49.783680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c384af9-65cb-4cdf-a540-9e1da1b0b1e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:49.783967 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c384af9-65cb-4cdf-a540-9e1da1b0b1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:49.784236 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c384af9-65cb-4cdf-a540-9e1da1b0b1e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:49.784958 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c384af9-65cb-4cdf-a540-9e1da1b0b1e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:49.788569 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c384af9-65cb-4cdf-a540-9e1da1b0b1e7 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:49.824649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1088/2157] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:04:49 2022] POST /identity/v3/projects => generated 389 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:04:49.829338 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f0247-fac6-40e1-8155-632e1231c7a6 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:49.841603 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f0247-fac6-40e1-8155-632e1231c7a6 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:49.931431 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9336b04-5630-4725-8db5-6fcad3bf2a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:49.932509 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9336b04-5630-4725-8db5-6fcad3bf2a5f admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:49.932766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9336b04-5630-4725-8db5-6fcad3bf2a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:49.933161 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9336b04-5630-4725-8db5-6fcad3bf2a5f admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:49.933968 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9336b04-5630-4725-8db5-6fcad3bf2a5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:49.936727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9336b04-5630-4725-8db5-6fcad3bf2a5f admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:49.964933 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1070/2158] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:04:49 2022] POST /identity/v3/users => generated 295 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:04:49.969654 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c384af9-65cb-4cdf-a540-9e1da1b0b1e7 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:49.982800 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c384af9-65cb-4cdf-a540-9e1da1b0b1e7 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:49.989606 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b616eb-d6ab-4c7c-ba92-16161fc7c0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:49.990242 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b616eb-d6ab-4c7c-ba92-16161fc7c0d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:49.990522 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b616eb-d6ab-4c7c-ba92-16161fc7c0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:49.990790 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b616eb-d6ab-4c7c-ba92-16161fc7c0d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:49.991490 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b616eb-d6ab-4c7c-ba92-16161fc7c0d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:49.993121 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b616eb-d6ab-4c7c-ba92-16161fc7c0d3 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:50.005484 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1089/2159] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:04:50.010679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9336b04-5630-4725-8db5-6fcad3bf2a5f admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:50.023034 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9336b04-5630-4725-8db5-6fcad3bf2a5f admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:50.029884 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f85a3-384c-42fa-916e-b3bf9fd7c1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:50.030682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f85a3-384c-42fa-916e-b3bf9fd7c1b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:50.030890 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f85a3-384c-42fa-916e-b3bf9fd7c1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:50.031158 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f85a3-384c-42fa-916e-b3bf9fd7c1b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:50.031855 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f85a3-384c-42fa-916e-b3bf9fd7c1b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:50.033480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f85a3-384c-42fa-916e-b3bf9fd7c1b1 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:50.045384 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1071/2160] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:04:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:04:50.049794 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b616eb-d6ab-4c7c-ba92-16161fc7c0d3 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:50.060976 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b616eb-d6ab-4c7c-ba92-16161fc7c0d3 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:50.065805 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899da451-1c0d-4a7c-a073-30b6a7a5c387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:50.066464 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899da451-1c0d-4a7c-a073-30b6a7a5c387 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:50.066747 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899da451-1c0d-4a7c-a073-30b6a7a5c387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:50.067032 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899da451-1c0d-4a7c-a073-30b6a7a5c387 admin admin] PATH_INFO: `/v3/projects/d6a3ff682a1b4a7687f29899d55ad680/users/61a673e6651c4c76af0037d8a17414b7/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:50.091911 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899da451-1c0d-4a7c-a073-30b6a7a5c387 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6a3ff682a1b4a7687f29899d55ad680, user_id=61a673e6651c4c76af0037d8a17414b7, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:50.094880 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899da451-1c0d-4a7c-a073-30b6a7a5c387 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:50.109775 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1090/2161] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:04:50 2022] PUT /identity/v3/projects/d6a3ff682a1b4a7687f29899d55ad680/users/61a673e6651c4c76af0037d8a17414b7/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:04:50.122891 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301dd62-374b-448e-8f9d-332c0281dc2a None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:50.123226 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301dd62-374b-448e-8f9d-332c0281dc2a None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:50.123447 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301dd62-374b-448e-8f9d-332c0281dc2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:50.166278 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-e301dd62-374b-448e-8f9d-332c0281dc2a None None] MFA Rules not processed for user `61a673e6651c4c76af0037d8a17414b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:04:50.167650 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-e301dd62-374b-448e-8f9d-332c0281dc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:50.255216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1072/2162] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:04:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:04:50.255973 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899da451-1c0d-4a7c-a073-30b6a7a5c387 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:50.257826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-899da451-1c0d-4a7c-a073-30b6a7a5c387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:50.323723 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899da451-1c0d-4a7c-a073-30b6a7a5c387 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:50.327749 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f849b3-fc2b-4afe-a903-217200461660 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:50.328372 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f849b3-fc2b-4afe-a903-217200461660 service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:50.328683 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f849b3-fc2b-4afe-a903-217200461660 service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:50.328957 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f849b3-fc2b-4afe-a903-217200461660 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:50.345779 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f849b3-fc2b-4afe-a903-217200461660 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:50.347464 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f849b3-fc2b-4afe-a903-217200461660 service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:50.359504 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1091/2163] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:04:50 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:04:51.299148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301dd62-374b-448e-8f9d-332c0281dc2a None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:51.313234 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301dd62-374b-448e-8f9d-332c0281dc2a None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:51.320178 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e64b8-6a48-4a2d-9d3a-3bbd5d854bc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:51.321085 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e64b8-6a48-4a2d-9d3a-3bbd5d854bc7 service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:51.321446 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e64b8-6a48-4a2d-9d3a-3bbd5d854bc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:51.321805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e64b8-6a48-4a2d-9d3a-3bbd5d854bc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:51.326606 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-3b7e64b8-6a48-4a2d-9d3a-3bbd5d854bc7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:51.421734 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7e64b8-6a48-4a2d-9d3a-3bbd5d854bc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:51.423311 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7e64b8-6a48-4a2d-9d3a-3bbd5d854bc7 service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:51.433731 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1073/2164] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:04:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:04:52.117894 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f849b3-fc2b-4afe-a903-217200461660 service cinder] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:52.127224 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f849b3-fc2b-4afe-a903-217200461660 service cinder] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:52.132689 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5693d-542d-4d8f-8983-209e03b8f102 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:52.132689 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5693d-542d-4d8f-8983-209e03b8f102 service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:52.132689 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5693d-542d-4d8f-8983-209e03b8f102 service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:52.133266 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5693d-542d-4d8f-8983-209e03b8f102 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:52.139453 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-12c5693d-542d-4d8f-8983-209e03b8f102 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:52.199326 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:52.272610 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c5693d-542d-4d8f-8983-209e03b8f102 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:52.274524 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c5693d-542d-4d8f-8983-209e03b8f102 service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:52.297012 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1092/2165] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:04:52 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:04:53.648369 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe82a8-74a7-4362-a147-48b1b6d268c3 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:53.649311 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe82a8-74a7-4362-a147-48b1b6d268c3 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:53.649983 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe82a8-74a7-4362-a147-48b1b6d268c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:53.659301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:53.682894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-13fe82a8-74a7-4362-a147-48b1b6d268c3 None None] MFA Rules not processed for user `160c88a7ba0144d4b1bc953767b12bb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:04:53.684508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-13fe82a8-74a7-4362-a147-48b1b6d268c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:53.779768 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1074/2166] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:04:53 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:04:53.786888 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5693d-542d-4d8f-8983-209e03b8f102 service cinder] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:53.788408 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-12c5693d-542d-4d8f-8983-209e03b8f102 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:53.875307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5693d-542d-4d8f-8983-209e03b8f102 service cinder] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:53.880940 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661eee9-d097-43e9-9a71-833d5a4ecab4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:53.881558 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661eee9-d097-43e9-9a71-833d5a4ecab4 service nova] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:53.881843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661eee9-d097-43e9-9a71-833d5a4ecab4 service nova] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:53.882120 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661eee9-d097-43e9-9a71-833d5a4ecab4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:53.891435 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661eee9-d097-43e9-9a71-833d5a4ecab4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:53.893174 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661eee9-d097-43e9-9a71-833d5a4ecab4 service nova] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:53.903748 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1093/2167] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:04:53 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:04:55.847294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2a827-4678-4184-ad9f-bc075580384c None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:55.847796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2a827-4678-4184-ad9f-bc075580384c None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:55.848005 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2a827-4678-4184-ad9f-bc075580384c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:55.872111 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-70e2a827-4678-4184-ad9f-bc075580384c None None] MFA Rules not processed for user `160c88a7ba0144d4b1bc953767b12bb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:04:55.873461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-70e2a827-4678-4184-ad9f-bc075580384c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:55.880539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1075/2168] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:04:55 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:05:03.898822 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=47 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:05:05.866300 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=33 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:05:24.799413 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661eee9-d097-43e9-9a71-833d5a4ecab4 service nova] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:24.800870 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-8661eee9-d097-43e9-9a71-833d5a4ecab4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:24.882568 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661eee9-d097-43e9-9a71-833d5a4ecab4 service nova] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:24.889417 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915bdd87-835e-4782-b050-e3d970323b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:24.890250 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915bdd87-835e-4782-b050-e3d970323b90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:24.890638 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915bdd87-835e-4782-b050-e3d970323b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:24.891020 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915bdd87-835e-4782-b050-e3d970323b90 admin admin] PATH_INFO: `/v3/users/aae3fd65389f4ec8933b0508ba677e50` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:24.892312 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915bdd87-835e-4782-b050-e3d970323b90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aae3fd65389f4ec8933b0508ba677e50)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:24.895348 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915bdd87-835e-4782-b050-e3d970323b90 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:24.965415 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-915bdd87-835e-4782-b050-e3d970323b90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aae3fd65389f4ec8933b0508ba677e50'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:24.973844 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-915bdd87-835e-4782-b050-e3d970323b90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aae3fd65389f4ec8933b0508ba677e50'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:24.984157 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-915bdd87-835e-4782-b050-e3d970323b90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aae3fd65389f4ec8933b0508ba677e50'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:25.006419 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-915bdd87-835e-4782-b050-e3d970323b90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aae3fd65389f4ec8933b0508ba677e50'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:25.007226 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-915bdd87-835e-4782-b050-e3d970323b90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aae3fd65389f4ec8933b0508ba677e50'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:25.018838 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1094/2169] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:05:24 2022] DELETE /identity/v3/users/aae3fd65389f4ec8933b0508ba677e50 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:05:25.020810 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4be7d-fad8-42f0-a3ec-2e6b3747edfe None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:25.021338 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4be7d-fad8-42f0-a3ec-2e6b3747edfe None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:25.021538 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4be7d-fad8-42f0-a3ec-2e6b3747edfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:25.059141 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-03e4be7d-fad8-42f0-a3ec-2e6b3747edfe None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:05:25.060393 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-03e4be7d-fad8-42f0-a3ec-2e6b3747edfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:25.138098 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1076/2170] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:05:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:05:25.148308 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915bdd87-835e-4782-b050-e3d970323b90 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:25.150504 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-915bdd87-835e-4782-b050-e3d970323b90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:25.226154 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915bdd87-835e-4782-b050-e3d970323b90 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:25.230458 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e9279-0c47-4e59-a764-eb02735a2207 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:25.231160 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e9279-0c47-4e59-a764-eb02735a2207 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:25.231500 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e9279-0c47-4e59-a764-eb02735a2207 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:25.231837 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e9279-0c47-4e59-a764-eb02735a2207 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:25.241891 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e9279-0c47-4e59-a764-eb02735a2207 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:25.243541 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e9279-0c47-4e59-a764-eb02735a2207 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:25.256118 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1095/2171] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:05:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:05:25.535573 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4be7d-fad8-42f0-a3ec-2e6b3747edfe None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:25.537426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-03e4be7d-fad8-42f0-a3ec-2e6b3747edfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:25.555694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4be7d-fad8-42f0-a3ec-2e6b3747edfe None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:25.562875 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650627c8-1ffa-4ab9-ade5-5e6ef0144d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:25.564529 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650627c8-1ffa-4ab9-ade5-5e6ef0144d08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:25.565607 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650627c8-1ffa-4ab9-ade5-5e6ef0144d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:25.566175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650627c8-1ffa-4ab9-ade5-5e6ef0144d08 admin admin] PATH_INFO: `/v3/projects/6a68616269204bec9023216f4d33c7b7` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:25.567571 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650627c8-1ffa-4ab9-ade5-5e6ef0144d08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a68616269204bec9023216f4d33c7b7)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:25.569635 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650627c8-1ffa-4ab9-ade5-5e6ef0144d08 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:25.627811 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-650627c8-1ffa-4ab9-ade5-5e6ef0144d08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a68616269204bec9023216f4d33c7b7', 'request_id': 'req-650627c8-1ffa-4ab9-ade5-5e6ef0144d08'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:25.639965 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-650627c8-1ffa-4ab9-ade5-5e6ef0144d08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a68616269204bec9023216f4d33c7b7', 'request_id': 'req-650627c8-1ffa-4ab9-ade5-5e6ef0144d08'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:25.646345 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-650627c8-1ffa-4ab9-ade5-5e6ef0144d08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a68616269204bec9023216f4d33c7b7', 'request_id': 'req-650627c8-1ffa-4ab9-ade5-5e6ef0144d08'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:25.657826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-650627c8-1ffa-4ab9-ade5-5e6ef0144d08 admin admin] The token cache is being invalidate because project 6a68616269204bec9023216f4d33c7b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:05:25.658107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-650627c8-1ffa-4ab9-ade5-5e6ef0144d08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:25.658736 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-650627c8-1ffa-4ab9-ade5-5e6ef0144d08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:25.660212 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1077/2172] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:05:25 2022] DELETE /identity/v3/projects/6a68616269204bec9023216f4d33c7b7 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:05:25.664196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e9279-0c47-4e59-a764-eb02735a2207 service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:25.674407 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e9279-0c47-4e59-a764-eb02735a2207 service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:25.692309 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:05:25.733039 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71953ffd-c915-47c5-b538-ef353f62a948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:25.733586 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71953ffd-c915-47c5-b538-ef353f62a948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:25.733864 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71953ffd-c915-47c5-b538-ef353f62a948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:25.734135 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71953ffd-c915-47c5-b538-ef353f62a948 admin admin] PATH_INFO: `/v3/users/031a09c7f7f24681a44c8025af34f75e` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:25.735287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71953ffd-c915-47c5-b538-ef353f62a948 admin admin] RBAC: Authorizing `identity:delete_user(user_id=031a09c7f7f24681a44c8025af34f75e)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:25.737239 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71953ffd-c915-47c5-b538-ef353f62a948 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:25.784453 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-71953ffd-c915-47c5-b538-ef353f62a948 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '031a09c7f7f24681a44c8025af34f75e'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:25.791861 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-71953ffd-c915-47c5-b538-ef353f62a948 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '031a09c7f7f24681a44c8025af34f75e'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:25.799618 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-71953ffd-c915-47c5-b538-ef353f62a948 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '031a09c7f7f24681a44c8025af34f75e'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:25.811703 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-71953ffd-c915-47c5-b538-ef353f62a948 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '031a09c7f7f24681a44c8025af34f75e'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:25.812354 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-71953ffd-c915-47c5-b538-ef353f62a948 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '031a09c7f7f24681a44c8025af34f75e'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:25.818116 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1096/2173] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:05:25 2022] DELETE /identity/v3/users/031a09c7f7f24681a44c8025af34f75e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:05:26.113676 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650627c8-1ffa-4ab9-ade5-5e6ef0144d08 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:26.116151 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-650627c8-1ffa-4ab9-ade5-5e6ef0144d08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:26.132081 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650627c8-1ffa-4ab9-ade5-5e6ef0144d08 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:26.202618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c69be-ac8e-4685-8090-55289a43fb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:26.202618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c69be-ac8e-4685-8090-55289a43fb37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:26.202618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c69be-ac8e-4685-8090-55289a43fb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:26.202618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c69be-ac8e-4685-8090-55289a43fb37 admin admin] PATH_INFO: `/v3/projects/fb987bbda16344edb1c01b700e112ac6` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:26.203003 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2c69be-ac8e-4685-8090-55289a43fb37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb987bbda16344edb1c01b700e112ac6)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:26.205040 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2c69be-ac8e-4685-8090-55289a43fb37 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:26.239452 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:05:26.259750 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-9b2c69be-ac8e-4685-8090-55289a43fb37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb987bbda16344edb1c01b700e112ac6', 'request_id': 'req-9b2c69be-ac8e-4685-8090-55289a43fb37'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:26.273176 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-9b2c69be-ac8e-4685-8090-55289a43fb37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb987bbda16344edb1c01b700e112ac6', 'request_id': 'req-9b2c69be-ac8e-4685-8090-55289a43fb37'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:26.283293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-9b2c69be-ac8e-4685-8090-55289a43fb37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb987bbda16344edb1c01b700e112ac6', 'request_id': 'req-9b2c69be-ac8e-4685-8090-55289a43fb37'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:26.294031 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-9b2c69be-ac8e-4685-8090-55289a43fb37 admin admin] The token cache is being invalidate because project fb987bbda16344edb1c01b700e112ac6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:05:26.294320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-9b2c69be-ac8e-4685-8090-55289a43fb37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:26.295549 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-9b2c69be-ac8e-4685-8090-55289a43fb37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:26.297419 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1078/2174] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:05:26 2022] DELETE /identity/v3/projects/fb987bbda16344edb1c01b700e112ac6 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:05:26.301223 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71953ffd-c915-47c5-b538-ef353f62a948 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:26.302637 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-71953ffd-c915-47c5-b538-ef353f62a948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:26.317353 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71953ffd-c915-47c5-b538-ef353f62a948 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:26.365916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c69be-ac8e-4685-8090-55289a43fb37 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:26.367509 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-9b2c69be-ac8e-4685-8090-55289a43fb37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:26.392222 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31087949-8ed5-49a3-a9f4-ea46914436bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:26.393268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31087949-8ed5-49a3-a9f4-ea46914436bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:26.393819 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31087949-8ed5-49a3-a9f4-ea46914436bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:26.393819 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31087949-8ed5-49a3-a9f4-ea46914436bb admin admin] PATH_INFO: `/v3/users/07e2532df79c49908a1ef68e37c22f98` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:26.394989 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31087949-8ed5-49a3-a9f4-ea46914436bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=07e2532df79c49908a1ef68e37c22f98)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:26.397267 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31087949-8ed5-49a3-a9f4-ea46914436bb admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:26.445887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c69be-ac8e-4685-8090-55289a43fb37 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:26.452003 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab937d22-a68d-4f46-b9c2-de9e8f00974c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:26.452887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab937d22-a68d-4f46-b9c2-de9e8f00974c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:26.453088 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab937d22-a68d-4f46-b9c2-de9e8f00974c admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:26.453414 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab937d22-a68d-4f46-b9c2-de9e8f00974c admin admin] PATH_INFO: `/v3/users/868b52d1f3494ece9fee62a577df6bc2` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:26.454791 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab937d22-a68d-4f46-b9c2-de9e8f00974c admin admin] RBAC: Authorizing `identity:delete_user(user_id=868b52d1f3494ece9fee62a577df6bc2)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:26.456726 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab937d22-a68d-4f46-b9c2-de9e8f00974c admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:26.458384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:05:26.461402 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-31087949-8ed5-49a3-a9f4-ea46914436bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07e2532df79c49908a1ef68e37c22f98'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:26.477500 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-31087949-8ed5-49a3-a9f4-ea46914436bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07e2532df79c49908a1ef68e37c22f98'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:26.485847 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-31087949-8ed5-49a3-a9f4-ea46914436bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07e2532df79c49908a1ef68e37c22f98'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:26.499994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-31087949-8ed5-49a3-a9f4-ea46914436bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07e2532df79c49908a1ef68e37c22f98'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:26.500667 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-31087949-8ed5-49a3-a9f4-ea46914436bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07e2532df79c49908a1ef68e37c22f98'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:26.506688 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1097/2175] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:05:26 2022] DELETE /identity/v3/users/07e2532df79c49908a1ef68e37c22f98 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:05:26.507650 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ab937d22-a68d-4f46-b9c2-de9e8f00974c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '868b52d1f3494ece9fee62a577df6bc2'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:26.515607 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ab937d22-a68d-4f46-b9c2-de9e8f00974c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '868b52d1f3494ece9fee62a577df6bc2'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:26.528940 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ab937d22-a68d-4f46-b9c2-de9e8f00974c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '868b52d1f3494ece9fee62a577df6bc2'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:26.529708 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ab937d22-a68d-4f46-b9c2-de9e8f00974c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '868b52d1f3494ece9fee62a577df6bc2'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:26.536603 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ab937d22-a68d-4f46-b9c2-de9e8f00974c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '868b52d1f3494ece9fee62a577df6bc2'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:26.543957 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1079/2176] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:05:26 2022] DELETE /identity/v3/users/868b52d1f3494ece9fee62a577df6bc2 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:05:26.548796 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49880b-83c2-4c56-8dab-1b0467531c3a None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:26.549119 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49880b-83c2-4c56-8dab-1b0467531c3a None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:26.549373 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49880b-83c2-4c56-8dab-1b0467531c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:26.570311 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-9e49880b-83c2-4c56-8dab-1b0467531c3a None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:05:26.571517 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-9e49880b-83c2-4c56-8dab-1b0467531c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:26.640532 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1098/2177] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:05:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:05:26.655777 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab937d22-a68d-4f46-b9c2-de9e8f00974c admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:26.657522 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-ab937d22-a68d-4f46-b9c2-de9e8f00974c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:26.734948 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:05:26.771972 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab937d22-a68d-4f46-b9c2-de9e8f00974c admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:26.778493 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a1dbeb-076a-4566-bd89-c8019ee4892f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:26.779201 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1dbeb-076a-4566-bd89-c8019ee4892f service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:26.779552 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1dbeb-076a-4566-bd89-c8019ee4892f service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:26.781096 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1dbeb-076a-4566-bd89-c8019ee4892f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:26.802473 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a1dbeb-076a-4566-bd89-c8019ee4892f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:26.810697 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a1dbeb-076a-4566-bd89-c8019ee4892f service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:26.827065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1080/2178] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:05:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:05:26.893892 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e49880b-83c2-4c56-8dab-1b0467531c3a None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:26.895366 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-9e49880b-83c2-4c56-8dab-1b0467531c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:26.914995 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e49880b-83c2-4c56-8dab-1b0467531c3a None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:26.921559 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd5b729-cf6d-400e-bb8f-3cc088344692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:26.922233 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5b729-cf6d-400e-bb8f-3cc088344692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:26.922516 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5b729-cf6d-400e-bb8f-3cc088344692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:26.922790 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5b729-cf6d-400e-bb8f-3cc088344692 admin admin] PATH_INFO: `/v3/projects/cce2281b43494adfb0e707b570c30615` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:26.923976 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd5b729-cf6d-400e-bb8f-3cc088344692 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cce2281b43494adfb0e707b570c30615)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:26.925928 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd5b729-cf6d-400e-bb8f-3cc088344692 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:26.982135 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-1cd5b729-cf6d-400e-bb8f-3cc088344692 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cce2281b43494adfb0e707b570c30615', 'request_id': 'req-1cd5b729-cf6d-400e-bb8f-3cc088344692'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:26.989256 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-1cd5b729-cf6d-400e-bb8f-3cc088344692 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cce2281b43494adfb0e707b570c30615', 'request_id': 'req-1cd5b729-cf6d-400e-bb8f-3cc088344692'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:26.997973 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-1cd5b729-cf6d-400e-bb8f-3cc088344692 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cce2281b43494adfb0e707b570c30615', 'request_id': 'req-1cd5b729-cf6d-400e-bb8f-3cc088344692'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:27.010262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-1cd5b729-cf6d-400e-bb8f-3cc088344692 admin admin] The token cache is being invalidate because project cce2281b43494adfb0e707b570c30615 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:05:27.011258 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-1cd5b729-cf6d-400e-bb8f-3cc088344692 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:27.011910 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-1cd5b729-cf6d-400e-bb8f-3cc088344692 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:27.013501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1099/2179] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:05:26 2022] DELETE /identity/v3/projects/cce2281b43494adfb0e707b570c30615 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:05:27.018557 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdabcb0f-8523-467f-9dff-d23b8ef1643a None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:27.018843 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdabcb0f-8523-467f-9dff-d23b8ef1643a None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:27.019119 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdabcb0f-8523-467f-9dff-d23b8ef1643a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:27.040779 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-cdabcb0f-8523-467f-9dff-d23b8ef1643a None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:05:27.042150 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-cdabcb0f-8523-467f-9dff-d23b8ef1643a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:27.104331 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1081/2180] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:05:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:05:27.111970 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182be20a-e919-4b80-81a1-f977d51794cc None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:27.112262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182be20a-e919-4b80-81a1-f977d51794cc None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:27.112935 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182be20a-e919-4b80-81a1-f977d51794cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:27.139118 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-182be20a-e919-4b80-81a1-f977d51794cc None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:05:27.140563 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-182be20a-e919-4b80-81a1-f977d51794cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:27.149132 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1100/2181] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:05:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:05:27.154391 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdabcb0f-8523-467f-9dff-d23b8ef1643a None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:27.164952 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdabcb0f-8523-467f-9dff-d23b8ef1643a None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:27.165498 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182be20a-e919-4b80-81a1-f977d51794cc None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:27.166975 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-182be20a-e919-4b80-81a1-f977d51794cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:27.170092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f04c6d-c8ec-4a14-9401-2886b125800e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:27.170736 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f04c6d-c8ec-4a14-9401-2886b125800e admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:27.171032 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f04c6d-c8ec-4a14-9401-2886b125800e admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:27.171296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f04c6d-c8ec-4a14-9401-2886b125800e admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:27.171967 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f04c6d-c8ec-4a14-9401-2886b125800e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:05:27.172343 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f04c6d-c8ec-4a14-9401-2886b125800e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:27.174079 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f04c6d-c8ec-4a14-9401-2886b125800e admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:27.182263 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182be20a-e919-4b80-81a1-f977d51794cc None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:27.187486 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1082/2182] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:05:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:05:27.191541 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f04c6d-c8ec-4a14-9401-2886b125800e admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:27.193043 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af634b7b-d473-4f29-9480-aeee6375392e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:27.193787 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af634b7b-d473-4f29-9480-aeee6375392e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:27.193985 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af634b7b-d473-4f29-9480-aeee6375392e admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:27.194264 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af634b7b-d473-4f29-9480-aeee6375392e admin admin] PATH_INFO: `/v3/projects/0968f83baf04490aaed1cddc81fc0d11` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:27.195424 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af634b7b-d473-4f29-9480-aeee6375392e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0968f83baf04490aaed1cddc81fc0d11)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:27.202868 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f04c6d-c8ec-4a14-9401-2886b125800e admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:27.207753 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7ede5-e50a-4d60-a3be-e6772f344eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:27.208359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7ede5-e50a-4d60-a3be-e6772f344eeb admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:27.208493 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af634b7b-d473-4f29-9480-aeee6375392e admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:27.208717 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7ede5-e50a-4d60-a3be-e6772f344eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:27.208993 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7ede5-e50a-4d60-a3be-e6772f344eeb admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:27.209676 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c7ede5-e50a-4d60-a3be-e6772f344eeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:27.211568 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c7ede5-e50a-4d60-a3be-e6772f344eeb admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:27.248965 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1083/2183] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:05:27 2022] POST /identity/v3/projects => generated 369 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:05:27.249956 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7ede5-e50a-4d60-a3be-e6772f344eeb admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:27.251874 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:05:27.269037 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-af634b7b-d473-4f29-9480-aeee6375392e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0968f83baf04490aaed1cddc81fc0d11', 'request_id': 'req-af634b7b-d473-4f29-9480-aeee6375392e'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:27.279901 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-af634b7b-d473-4f29-9480-aeee6375392e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0968f83baf04490aaed1cddc81fc0d11', 'request_id': 'req-af634b7b-d473-4f29-9480-aeee6375392e'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:27.290689 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-af634b7b-d473-4f29-9480-aeee6375392e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0968f83baf04490aaed1cddc81fc0d11', 'request_id': 'req-af634b7b-d473-4f29-9480-aeee6375392e'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:27.306449 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-af634b7b-d473-4f29-9480-aeee6375392e admin admin] The token cache is being invalidate because project 0968f83baf04490aaed1cddc81fc0d11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:05:27.306742 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-af634b7b-d473-4f29-9480-aeee6375392e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:27.308081 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-af634b7b-d473-4f29-9480-aeee6375392e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:27.309550 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1101/2184] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:05:27 2022] DELETE /identity/v3/projects/0968f83baf04490aaed1cddc81fc0d11 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:05:27.317642 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af634b7b-d473-4f29-9480-aeee6375392e admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:27.319488 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-af634b7b-d473-4f29-9480-aeee6375392e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:27.325481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7ede5-e50a-4d60-a3be-e6772f344eeb admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:27.375327 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:05:27.389018 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46953732-b04e-463b-916b-2e728bcc7937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:27.389655 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46953732-b04e-463b-916b-2e728bcc7937 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:27.389931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46953732-b04e-463b-916b-2e728bcc7937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:27.390200 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46953732-b04e-463b-916b-2e728bcc7937 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:27.390908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46953732-b04e-463b-916b-2e728bcc7937 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:27.393121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46953732-b04e-463b-916b-2e728bcc7937 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:27.408616 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af634b7b-d473-4f29-9480-aeee6375392e admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:27.413337 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d4ca3-a952-4449-8bc5-0f8f9e44c45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:27.413981 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d4ca3-a952-4449-8bc5-0f8f9e44c45b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:27.414284 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d4ca3-a952-4449-8bc5-0f8f9e44c45b admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:27.414546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d4ca3-a952-4449-8bc5-0f8f9e44c45b admin admin] PATH_INFO: `/v3/users/791b476baae94dc395131fd1ca5010b0` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:27.415580 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1084/2185] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:05:27 2022] POST /identity/v3/users => generated 285 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:05:27.415818 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d4ca3-a952-4449-8bc5-0f8f9e44c45b admin admin] RBAC: Authorizing `identity:delete_user(user_id=791b476baae94dc395131fd1ca5010b0)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:27.417945 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d4ca3-a952-4449-8bc5-0f8f9e44c45b admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:27.419958 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46953732-b04e-463b-916b-2e728bcc7937 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:27.421689 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-46953732-b04e-463b-916b-2e728bcc7937 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:27.435941 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46953732-b04e-463b-916b-2e728bcc7937 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:27.441159 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0795cafd-b790-4c73-b010-4dbe550054fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:27.441871 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795cafd-b790-4c73-b010-4dbe550054fd admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:27.442154 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795cafd-b790-4c73-b010-4dbe550054fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:27.442426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795cafd-b790-4c73-b010-4dbe550054fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:27.443125 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0795cafd-b790-4c73-b010-4dbe550054fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:27.444754 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0795cafd-b790-4c73-b010-4dbe550054fd admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:27.460796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1085/2186] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:05:27.464605 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0795cafd-b790-4c73-b010-4dbe550054fd admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:27.477119 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0795cafd-b790-4c73-b010-4dbe550054fd admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:27.480424 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-355d4ca3-a952-4449-8bc5-0f8f9e44c45b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '791b476baae94dc395131fd1ca5010b0'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:27.492236 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625570c3-8674-453c-9161-9fedcd94b111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:27.494005 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625570c3-8674-453c-9161-9fedcd94b111 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:27.494288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625570c3-8674-453c-9161-9fedcd94b111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:27.494558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625570c3-8674-453c-9161-9fedcd94b111 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:27.496086 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625570c3-8674-453c-9161-9fedcd94b111 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:27.498019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625570c3-8674-453c-9161-9fedcd94b111 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:27.499652 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-355d4ca3-a952-4449-8bc5-0f8f9e44c45b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '791b476baae94dc395131fd1ca5010b0'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:27.509194 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-355d4ca3-a952-4449-8bc5-0f8f9e44c45b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '791b476baae94dc395131fd1ca5010b0'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:27.513380 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1086/2187] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:05:27.514511 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625570c3-8674-453c-9161-9fedcd94b111 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:27.527530 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625570c3-8674-453c-9161-9fedcd94b111 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:27.550290 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4d4dd-1f61-4582-8ef8-0d4d546db8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:27.551266 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4d4dd-1f61-4582-8ef8-0d4d546db8b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:27.551567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4d4dd-1f61-4582-8ef8-0d4d546db8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:27.551795 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4d4dd-1f61-4582-8ef8-0d4d546db8b2 admin admin] PATH_INFO: `/v3/projects/08cb43e2c9c84cffa058389e8fa59dd9/users/2d8b9ca1937c41af8c34176f589e44a8/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:27.552157 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-355d4ca3-a952-4449-8bc5-0f8f9e44c45b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '791b476baae94dc395131fd1ca5010b0'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:27.552944 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-355d4ca3-a952-4449-8bc5-0f8f9e44c45b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '791b476baae94dc395131fd1ca5010b0'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:27.569503 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1102/2188] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:05:27 2022] DELETE /identity/v3/users/791b476baae94dc395131fd1ca5010b0 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:05:27.570960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4d4dd-1f61-4582-8ef8-0d4d546db8b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08cb43e2c9c84cffa058389e8fa59dd9, user_id=2d8b9ca1937c41af8c34176f589e44a8, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:27.574325 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4d4dd-1f61-4582-8ef8-0d4d546db8b2 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:27.595473 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1087/2189] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:05:27 2022] PUT /identity/v3/projects/08cb43e2c9c84cffa058389e8fa59dd9/users/2d8b9ca1937c41af8c34176f589e44a8/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:05:27.606859 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da97123-13f7-4d6b-b4e8-ee21310de7a4 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:27.607322 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da97123-13f7-4d6b-b4e8-ee21310de7a4 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:27.607677 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da97123-13f7-4d6b-b4e8-ee21310de7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:27.646409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-1da97123-13f7-4d6b-b4e8-ee21310de7a4 None None] MFA Rules not processed for user `2d8b9ca1937c41af8c34176f589e44a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:05:27.647824 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-1da97123-13f7-4d6b-b4e8-ee21310de7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:27.693323 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:05:27.743039 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1103/2190] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:05:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:05:27.755622 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4d4dd-1f61-4582-8ef8-0d4d546db8b2 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:27.757590 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-6ac4d4dd-1f61-4582-8ef8-0d4d546db8b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:27.828108 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4d4dd-1f61-4582-8ef8-0d4d546db8b2 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:27.836272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aaf33d-eefd-42ae-8c05-ad8aea124f76 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe69b398c1f4e18b625a63ae40b01e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:27.837017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aaf33d-eefd-42ae-8c05-ad8aea124f76 service glance] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:27.837346 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aaf33d-eefd-42ae-8c05-ad8aea124f76 service glance] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:27.837685 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aaf33d-eefd-42ae-8c05-ad8aea124f76 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:27.851109 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aaf33d-eefd-42ae-8c05-ad8aea124f76 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:27.852380 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aaf33d-eefd-42ae-8c05-ad8aea124f76 service glance] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:27.863302 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1088/2191] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:05:27 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:05:27.907927 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da97123-13f7-4d6b-b4e8-ee21310de7a4 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:27.909401 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-1da97123-13f7-4d6b-b4e8-ee21310de7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:28.004064 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da97123-13f7-4d6b-b4e8-ee21310de7a4 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:28.008990 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac657c-f5d1-4ada-86d9-c4b5f745e6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:28.009663 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac657c-f5d1-4ada-86d9-c4b5f745e6a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:28.009972 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac657c-f5d1-4ada-86d9-c4b5f745e6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:28.010249 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac657c-f5d1-4ada-86d9-c4b5f745e6a9 admin admin] PATH_INFO: `/v3/projects/d5d516c7152848fab0fd7d874d019db1` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:28.012073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ac657c-f5d1-4ada-86d9-c4b5f745e6a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5d516c7152848fab0fd7d874d019db1)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:28.014247 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ac657c-f5d1-4ada-86d9-c4b5f745e6a9 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:28.069835 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-f1ac657c-f5d1-4ada-86d9-c4b5f745e6a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5d516c7152848fab0fd7d874d019db1', 'request_id': 'req-f1ac657c-f5d1-4ada-86d9-c4b5f745e6a9'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:28.077111 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-f1ac657c-f5d1-4ada-86d9-c4b5f745e6a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5d516c7152848fab0fd7d874d019db1', 'request_id': 'req-f1ac657c-f5d1-4ada-86d9-c4b5f745e6a9'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:28.086221 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-f1ac657c-f5d1-4ada-86d9-c4b5f745e6a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5d516c7152848fab0fd7d874d019db1', 'request_id': 'req-f1ac657c-f5d1-4ada-86d9-c4b5f745e6a9'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:28.100839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-f1ac657c-f5d1-4ada-86d9-c4b5f745e6a9 admin admin] The token cache is being invalidate because project d5d516c7152848fab0fd7d874d019db1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:05:28.100839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-f1ac657c-f5d1-4ada-86d9-c4b5f745e6a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:28.101174 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-f1ac657c-f5d1-4ada-86d9-c4b5f745e6a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:28.102886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1104/2192] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:05:27 2022] DELETE /identity/v3/projects/d5d516c7152848fab0fd7d874d019db1 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:05:28.107685 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3b032-6a55-4485-a6de-1d3b48cab3a0 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:28.107996 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3b032-6a55-4485-a6de-1d3b48cab3a0 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:28.108237 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3b032-6a55-4485-a6de-1d3b48cab3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:28.129422 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-ede3b032-6a55-4485-a6de-1d3b48cab3a0 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:05:28.130606 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-ede3b032-6a55-4485-a6de-1d3b48cab3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:28.227252 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1089/2193] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:05:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:05:28.237515 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9b449-9891-4043-85a2-ddbfee480ed0 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:28.237515 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9b449-9891-4043-85a2-ddbfee480ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:28.237515 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9b449-9891-4043-85a2-ddbfee480ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:28.259434 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-c0f9b449-9891-4043-85a2-ddbfee480ed0 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:05:28.260819 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-c0f9b449-9891-4043-85a2-ddbfee480ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:28.270989 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1105/2194] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:05:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:05:28.275511 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3b032-6a55-4485-a6de-1d3b48cab3a0 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:28.289195 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3b032-6a55-4485-a6de-1d3b48cab3a0 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:28.295875 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e36e9-89ce-466e-8e6a-ab1626f8a43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:28.296489 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e36e9-89ce-466e-8e6a-ab1626f8a43e admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:28.297109 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e36e9-89ce-466e-8e6a-ab1626f8a43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:28.297255 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e36e9-89ce-466e-8e6a-ab1626f8a43e admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:28.298098 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e36e9-89ce-466e-8e6a-ab1626f8a43e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:05:28.298357 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e36e9-89ce-466e-8e6a-ab1626f8a43e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:28.300153 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e36e9-89ce-466e-8e6a-ab1626f8a43e admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:28.313971 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1090/2195] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:05:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:05:28.318469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f9b449-9891-4043-85a2-ddbfee480ed0 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:28.328594 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f9b449-9891-4043-85a2-ddbfee480ed0 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:28.334742 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003373e-28a4-4329-b760-1a8f2f1a452b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:28.335506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003373e-28a4-4329-b760-1a8f2f1a452b admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:28.335854 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003373e-28a4-4329-b760-1a8f2f1a452b admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:28.336142 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003373e-28a4-4329-b760-1a8f2f1a452b admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:28.337005 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a003373e-28a4-4329-b760-1a8f2f1a452b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:28.339080 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a003373e-28a4-4329-b760-1a8f2f1a452b admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:28.376567 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1106/2196] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:05:28 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:05:28.377745 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e36e9-89ce-466e-8e6a-ab1626f8a43e admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:28.424916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e36e9-89ce-466e-8e6a-ab1626f8a43e admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:28.513162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:05:28.569418 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48d5a3-6bd4-4329-a353-e40f9e463ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:28.570042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48d5a3-6bd4-4329-a353-e40f9e463ca8 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:28.570323 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48d5a3-6bd4-4329-a353-e40f9e463ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:28.570601 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48d5a3-6bd4-4329-a353-e40f9e463ca8 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:28.571337 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa48d5a3-6bd4-4329-a353-e40f9e463ca8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:28.573844 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa48d5a3-6bd4-4329-a353-e40f9e463ca8 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:28.596329 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1091/2197] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:05:28 2022] POST /identity/v3/users => generated 288 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:05:28.600124 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003373e-28a4-4329-b760-1a8f2f1a452b admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:28.611611 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003373e-28a4-4329-b760-1a8f2f1a452b admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:28.616313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23edbe-8e3e-43d0-a73c-f210e0c22ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:28.617036 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23edbe-8e3e-43d0-a73c-f210e0c22ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:28.617405 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23edbe-8e3e-43d0-a73c-f210e0c22ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:28.617754 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23edbe-8e3e-43d0-a73c-f210e0c22ed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:28.618519 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23edbe-8e3e-43d0-a73c-f210e0c22ed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:28.621145 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23edbe-8e3e-43d0-a73c-f210e0c22ed1 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:28.632309 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1107/2198] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:05:28.636390 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48d5a3-6bd4-4329-a353-e40f9e463ca8 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:28.646821 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48d5a3-6bd4-4329-a353-e40f9e463ca8 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:28.652905 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f3686-2a26-40c1-8815-7d75bd7014a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:28.653621 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f3686-2a26-40c1-8815-7d75bd7014a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:28.653915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f3686-2a26-40c1-8815-7d75bd7014a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:28.654184 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f3686-2a26-40c1-8815-7d75bd7014a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:28.655432 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f3686-2a26-40c1-8815-7d75bd7014a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:28.656485 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f3686-2a26-40c1-8815-7d75bd7014a1 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:28.667826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1092/2199] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:05:28.672040 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23edbe-8e3e-43d0-a73c-f210e0c22ed1 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:28.683018 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23edbe-8e3e-43d0-a73c-f210e0c22ed1 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:28.687585 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d8e16-c672-46fb-9325-e670f1b55db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:28.688242 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d8e16-c672-46fb-9325-e670f1b55db0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:28.688551 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d8e16-c672-46fb-9325-e670f1b55db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:28.689041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d8e16-c672-46fb-9325-e670f1b55db0 admin admin] PATH_INFO: `/v3/projects/04eee2a554704828ba94f8acb136d169/users/d5854e86e43b493dbe722d8b9f8581c3/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:28.706293 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d8e16-c672-46fb-9325-e670f1b55db0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04eee2a554704828ba94f8acb136d169, user_id=d5854e86e43b493dbe722d8b9f8581c3, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:28.709063 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d8e16-c672-46fb-9325-e670f1b55db0 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:28.714896 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:05:28.727745 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1108/2200] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:05:28 2022] PUT /identity/v3/projects/04eee2a554704828ba94f8acb136d169/users/d5854e86e43b493dbe722d8b9f8581c3/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:05:28.741197 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f54c90-f5f3-433b-9fb1-acfd63e56e59 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:28.741493 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f54c90-f5f3-433b-9fb1-acfd63e56e59 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:28.741756 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f54c90-f5f3-433b-9fb1-acfd63e56e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:28.775050 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-e2f54c90-f5f3-433b-9fb1-acfd63e56e59 None None] MFA Rules not processed for user `d5854e86e43b493dbe722d8b9f8581c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:05:28.777828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-e2f54c90-f5f3-433b-9fb1-acfd63e56e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:28.863288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1093/2201] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:05:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:05:28.872536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d8e16-c672-46fb-9325-e670f1b55db0 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:28.954810 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d8e16-c672-46fb-9325-e670f1b55db0 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:28.961805 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4bfde7-815e-46a7-8c2c-80a289df9524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:28.962663 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4bfde7-815e-46a7-8c2c-80a289df9524 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:28.962989 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4bfde7-815e-46a7-8c2c-80a289df9524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:28.963291 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4bfde7-815e-46a7-8c2c-80a289df9524 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:28.964300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4bfde7-815e-46a7-8c2c-80a289df9524 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:28.967721 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4bfde7-815e-46a7-8c2c-80a289df9524 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:29.026610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f54c90-f5f3-433b-9fb1-acfd63e56e59 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:29.028745 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1109/2202] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:05:28 2022] POST /identity/v3/projects => generated 375 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:05:29.038769 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f54c90-f5f3-433b-9fb1-acfd63e56e59 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:29.127709 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2b7b1-32ce-4c21-9eff-2bbc0edee07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:29.129335 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2b7b1-32ce-4c21-9eff-2bbc0edee07a admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:29.129696 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2b7b1-32ce-4c21-9eff-2bbc0edee07a admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:29.130104 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2b7b1-32ce-4c21-9eff-2bbc0edee07a admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:29.130854 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb2b7b1-32ce-4c21-9eff-2bbc0edee07a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:29.139349 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb2b7b1-32ce-4c21-9eff-2bbc0edee07a admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:29.148800 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4bfde7-815e-46a7-8c2c-80a289df9524 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:29.150506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-ef4bfde7-815e-46a7-8c2c-80a289df9524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:29.165583 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4bfde7-815e-46a7-8c2c-80a289df9524 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:29.171463 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14026e20-4095-4789-8f4d-4e43a86e9765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:29.172629 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14026e20-4095-4789-8f4d-4e43a86e9765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:29.172629 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14026e20-4095-4789-8f4d-4e43a86e9765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:29.172877 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14026e20-4095-4789-8f4d-4e43a86e9765 admin admin] PATH_INFO: `/v3/users/61a673e6651c4c76af0037d8a17414b7` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:29.174244 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14026e20-4095-4789-8f4d-4e43a86e9765 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61a673e6651c4c76af0037d8a17414b7)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:29.176330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14026e20-4095-4789-8f4d-4e43a86e9765 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:29.192628 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1094/2203] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:05:29 2022] POST /identity/v3/users => generated 294 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:05:29.195176 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2b7b1-32ce-4c21-9eff-2bbc0edee07a admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:29.214317 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2b7b1-32ce-4c21-9eff-2bbc0edee07a admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:29.223154 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b0525-0f74-4ddf-8c3d-d9d2b63993bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:29.223788 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b0525-0f74-4ddf-8c3d-d9d2b63993bd admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:29.224072 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b0525-0f74-4ddf-8c3d-d9d2b63993bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:29.224411 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b0525-0f74-4ddf-8c3d-d9d2b63993bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:29.225288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b0525-0f74-4ddf-8c3d-d9d2b63993bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:29.227119 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b0525-0f74-4ddf-8c3d-d9d2b63993bd admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:29.244705 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-14026e20-4095-4789-8f4d-4e43a86e9765 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61a673e6651c4c76af0037d8a17414b7'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:29.252835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1095/2204] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:05:29.255318 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-14026e20-4095-4789-8f4d-4e43a86e9765 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61a673e6651c4c76af0037d8a17414b7'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:29.257093 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b0525-0f74-4ddf-8c3d-d9d2b63993bd admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:29.266275 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-14026e20-4095-4789-8f4d-4e43a86e9765 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61a673e6651c4c76af0037d8a17414b7'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:29.271234 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b0525-0f74-4ddf-8c3d-d9d2b63993bd admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:29.284370 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-14026e20-4095-4789-8f4d-4e43a86e9765 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61a673e6651c4c76af0037d8a17414b7'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:29.285354 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-14026e20-4095-4789-8f4d-4e43a86e9765 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61a673e6651c4c76af0037d8a17414b7'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:29.291113 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff1b49-275b-4079-8fca-45719b01c1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:29.292065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff1b49-275b-4079-8fca-45719b01c1a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:29.292356 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff1b49-275b-4079-8fca-45719b01c1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:29.292855 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff1b49-275b-4079-8fca-45719b01c1a2 admin admin] PATH_INFO: `/v3/projects/1302c1f6c5b1466581bc72483e5955ec/users/3476c4f1de754f7ea5afb4e727f834fa/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:29.295249 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1110/2205] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:05:29 2022] DELETE /identity/v3/users/61a673e6651c4c76af0037d8a17414b7 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:05:29.300176 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10842f4-a8bc-4de8-add7-bfff4383e468 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:29.300509 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10842f4-a8bc-4de8-add7-bfff4383e468 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:29.300815 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10842f4-a8bc-4de8-add7-bfff4383e468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:29.300928 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:05:29.331434 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff1b49-275b-4079-8fca-45719b01c1a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1302c1f6c5b1466581bc72483e5955ec, user_id=3476c4f1de754f7ea5afb4e727f834fa, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:29.334131 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-b10842f4-a8bc-4de8-add7-bfff4383e468 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:05:29.335380 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-b10842f4-a8bc-4de8-add7-bfff4383e468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:29.335981 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff1b49-275b-4079-8fca-45719b01c1a2 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:29.356615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1096/2206] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:05:29 2022] PUT /identity/v3/projects/1302c1f6c5b1466581bc72483e5955ec/users/3476c4f1de754f7ea5afb4e727f834fa/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:05:29.359288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff1b49-275b-4079-8fca-45719b01c1a2 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:29.362346 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-52ff1b49-275b-4079-8fca-45719b01c1a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:29.401094 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:05:29.420820 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1111/2207] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:05:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:05:29.428044 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10842f4-a8bc-4de8-add7-bfff4383e468 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:29.429882 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-b10842f4-a8bc-4de8-add7-bfff4383e468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:29.450431 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff1b49-275b-4079-8fca-45719b01c1a2 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:29.460485 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc6fdc-3e1a-4d57-bd1d-f2fc01342a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:29.460485 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc6fdc-3e1a-4d57-bd1d-f2fc01342a06 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:29.460485 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc6fdc-3e1a-4d57-bd1d-f2fc01342a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:29.460485 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc6fdc-3e1a-4d57-bd1d-f2fc01342a06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:29.460976 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc6fdc-3e1a-4d57-bd1d-f2fc01342a06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:29.460976 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc6fdc-3e1a-4d57-bd1d-f2fc01342a06 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:29.476818 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1097/2208] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:05:29.478556 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc6fdc-3e1a-4d57-bd1d-f2fc01342a06 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:29.490383 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc6fdc-3e1a-4d57-bd1d-f2fc01342a06 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:29.497124 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3581920f-3176-4f48-acec-f15cf63fccbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:29.497810 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581920f-3176-4f48-acec-f15cf63fccbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:29.498053 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581920f-3176-4f48-acec-f15cf63fccbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:29.498369 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581920f-3176-4f48-acec-f15cf63fccbe admin admin] PATH_INFO: `/v3/projects/1302c1f6c5b1466581bc72483e5955ec/users/3476c4f1de754f7ea5afb4e727f834fa/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:29.500751 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3581920f-3176-4f48-acec-f15cf63fccbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=1302c1f6c5b1466581bc72483e5955ec, user_id=3476c4f1de754f7ea5afb4e727f834fa, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:29.507794 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3581920f-3176-4f48-acec-f15cf63fccbe admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:29.508608 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10842f4-a8bc-4de8-add7-bfff4383e468 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:29.520756 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1098/2209] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:05:29 2022] PUT /identity/v3/projects/1302c1f6c5b1466581bc72483e5955ec/users/3476c4f1de754f7ea5afb4e727f834fa/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:05:29.541304 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a608661-df61-4b88-b89b-38b266603d4a None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:29.542012 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a608661-df61-4b88-b89b-38b266603d4a None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:29.542443 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a608661-df61-4b88-b89b-38b266603d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:29.578977 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-6a608661-df61-4b88-b89b-38b266603d4a None None] MFA Rules not processed for user `3476c4f1de754f7ea5afb4e727f834fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:05:29.580409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-6a608661-df61-4b88-b89b-38b266603d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:29.594790 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ca083-ab74-4723-abf6-d7c85c4c49a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:29.595955 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ca083-ab74-4723-abf6-d7c85c4c49a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:29.595955 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ca083-ab74-4723-abf6-d7c85c4c49a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:29.596205 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ca083-ab74-4723-abf6-d7c85c4c49a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:29.620674 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ca083-ab74-4723-abf6-d7c85c4c49a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:29.622198 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ca083-ab74-4723-abf6-d7c85c4c49a0 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:29.633097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1112/2210] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:05:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:05:29.684801 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1099/2211] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:05:29 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:05:29.695595 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ca083-ab74-4723-abf6-d7c85c4c49a0 service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:29.699168 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-383ca083-ab74-4723-abf6-d7c85c4c49a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:29.766832 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ca083-ab74-4723-abf6-d7c85c4c49a0 service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:29.840153 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaeec17-e13e-4bfc-87f9-37912653c3c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:29.841002 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaeec17-e13e-4bfc-87f9-37912653c3c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:29.841251 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaeec17-e13e-4bfc-87f9-37912653c3c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:29.841654 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaeec17-e13e-4bfc-87f9-37912653c3c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:29.845923 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-adaeec17-e13e-4bfc-87f9-37912653c3c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:29.869055 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:05:29.916843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaeec17-e13e-4bfc-87f9-37912653c3c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:29.918941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaeec17-e13e-4bfc-87f9-37912653c3c5 service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:29.928869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1113/2212] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:05:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:05:30.065085 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a608661-df61-4b88-b89b-38b266603d4a None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:30.066680 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-6a608661-df61-4b88-b89b-38b266603d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:30.164196 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:05:30.180291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a608661-df61-4b88-b89b-38b266603d4a None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:30.185757 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9dc210-ff93-4711-bc19-616ee0610a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:30.186402 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9dc210-ff93-4711-bc19-616ee0610a99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:30.186859 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9dc210-ff93-4711-bc19-616ee0610a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:30.187102 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9dc210-ff93-4711-bc19-616ee0610a99 admin admin] PATH_INFO: `/v3/projects/d6a3ff682a1b4a7687f29899d55ad680` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:30.188897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9dc210-ff93-4711-bc19-616ee0610a99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6a3ff682a1b4a7687f29899d55ad680)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:30.191138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9dc210-ff93-4711-bc19-616ee0610a99 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:30.246699 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-4d9dc210-ff93-4711-bc19-616ee0610a99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6a3ff682a1b4a7687f29899d55ad680', 'request_id': 'req-4d9dc210-ff93-4711-bc19-616ee0610a99'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:30.257704 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-4d9dc210-ff93-4711-bc19-616ee0610a99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6a3ff682a1b4a7687f29899d55ad680', 'request_id': 'req-4d9dc210-ff93-4711-bc19-616ee0610a99'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:30.266461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-4d9dc210-ff93-4711-bc19-616ee0610a99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6a3ff682a1b4a7687f29899d55ad680', 'request_id': 'req-4d9dc210-ff93-4711-bc19-616ee0610a99'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:30.286743 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-4d9dc210-ff93-4711-bc19-616ee0610a99 admin admin] The token cache is being invalidate because project d6a3ff682a1b4a7687f29899d55ad680 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:05:30.287027 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-4d9dc210-ff93-4711-bc19-616ee0610a99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:30.287834 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-4d9dc210-ff93-4711-bc19-616ee0610a99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:05:30.289608 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1100/2213] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:05:30 2022] DELETE /identity/v3/projects/d6a3ff682a1b4a7687f29899d55ad680 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:05:30.295357 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0650bba-749e-43d2-bed4-b5c7aa760411 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:30.296127 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0650bba-749e-43d2-bed4-b5c7aa760411 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:30.296616 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0650bba-749e-43d2-bed4-b5c7aa760411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:30.323073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-c0650bba-749e-43d2-bed4-b5c7aa760411 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:05:30.332851 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-c0650bba-749e-43d2-bed4-b5c7aa760411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:30.401233 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1114/2214] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:05:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:05:30.410209 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ec1d1-ca43-40f8-a9c5-b787e041ebb0 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:30.410495 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ec1d1-ca43-40f8-a9c5-b787e041ebb0 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:30.410775 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ec1d1-ca43-40f8-a9c5-b787e041ebb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:30.431851 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-864ec1d1-ca43-40f8-a9c5-b787e041ebb0 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:05:30.434472 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-864ec1d1-ca43-40f8-a9c5-b787e041ebb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:30.444661 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1101/2215] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:05:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:05:30.447758 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0650bba-749e-43d2-bed4-b5c7aa760411 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:30.459561 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0650bba-749e-43d2-bed4-b5c7aa760411 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:30.465140 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ccd39-01e4-410a-8007-223ce5a2a4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:30.465898 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ccd39-01e4-410a-8007-223ce5a2a4bb admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:30.469172 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ccd39-01e4-410a-8007-223ce5a2a4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:30.469379 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ccd39-01e4-410a-8007-223ce5a2a4bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:30.470110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152ccd39-01e4-410a-8007-223ce5a2a4bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:05:30.470428 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152ccd39-01e4-410a-8007-223ce5a2a4bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:30.472026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152ccd39-01e4-410a-8007-223ce5a2a4bb admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:30.487943 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1115/2216] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:05:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:05:30.491271 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ec1d1-ca43-40f8-a9c5-b787e041ebb0 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:30.501210 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ec1d1-ca43-40f8-a9c5-b787e041ebb0 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:30.505859 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf813e-60b9-46ae-9fc9-783e481f4b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:30.506462 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf813e-60b9-46ae-9fc9-783e481f4b66 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:30.506743 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf813e-60b9-46ae-9fc9-783e481f4b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:30.507015 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf813e-60b9-46ae-9fc9-783e481f4b66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:30.507670 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf813e-60b9-46ae-9fc9-783e481f4b66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:30.509615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf813e-60b9-46ae-9fc9-783e481f4b66 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:30.536832 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1102/2217] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:05:30 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:05:30.540944 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ccd39-01e4-410a-8007-223ce5a2a4bb admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:30.552130 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ccd39-01e4-410a-8007-223ce5a2a4bb admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:30.608942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9f6b2-0daf-439f-8e4f-cec7b239770c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:30.609790 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9f6b2-0daf-439f-8e4f-cec7b239770c admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:30.610102 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9f6b2-0daf-439f-8e4f-cec7b239770c admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:30.610373 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9f6b2-0daf-439f-8e4f-cec7b239770c admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:30.611050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c9f6b2-0daf-439f-8e4f-cec7b239770c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:30.613026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c9f6b2-0daf-439f-8e4f-cec7b239770c admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:30.642346 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1116/2218] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:05:30 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:05:30.646629 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf813e-60b9-46ae-9fc9-783e481f4b66 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:30.657391 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf813e-60b9-46ae-9fc9-783e481f4b66 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:30.662298 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e5f6a-9b06-4e29-b4e1-591c878ef238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:30.662929 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e5f6a-9b06-4e29-b4e1-591c878ef238 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:30.663212 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e5f6a-9b06-4e29-b4e1-591c878ef238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:30.663492 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e5f6a-9b06-4e29-b4e1-591c878ef238 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:30.664202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e5f6a-9b06-4e29-b4e1-591c878ef238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:30.665860 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e5f6a-9b06-4e29-b4e1-591c878ef238 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:30.680493 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1103/2219] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:05:30.681921 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9f6b2-0daf-439f-8e4f-cec7b239770c admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:30.694314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9f6b2-0daf-439f-8e4f-cec7b239770c admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:30.702013 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a8d5a-8195-4a94-b1d3-670d53ee8c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:30.702761 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a8d5a-8195-4a94-b1d3-670d53ee8c70 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:30.702953 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a8d5a-8195-4a94-b1d3-670d53ee8c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:30.703203 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a8d5a-8195-4a94-b1d3-670d53ee8c70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:30.703907 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5a8d5a-8195-4a94-b1d3-670d53ee8c70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:30.705613 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5a8d5a-8195-4a94-b1d3-670d53ee8c70 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:30.719416 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1117/2220] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:05:30.723959 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e5f6a-9b06-4e29-b4e1-591c878ef238 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:30.736317 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e5f6a-9b06-4e29-b4e1-591c878ef238 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:30.741325 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff3b06-3856-4cbe-abe0-eaeee0969529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:30.741984 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff3b06-3856-4cbe-abe0-eaeee0969529 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:30.742267 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff3b06-3856-4cbe-abe0-eaeee0969529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:30.742541 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff3b06-3856-4cbe-abe0-eaeee0969529 admin admin] PATH_INFO: `/v3/projects/3f8a529448f14ef18a72de884f17d0f6/users/0e022456457d41b8812058252e4a43f4/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:30.759919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ff3b06-3856-4cbe-abe0-eaeee0969529 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f8a529448f14ef18a72de884f17d0f6, user_id=0e022456457d41b8812058252e4a43f4, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:30.762709 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ff3b06-3856-4cbe-abe0-eaeee0969529 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:30.778160 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1104/2221] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:05:30 2022] PUT /identity/v3/projects/3f8a529448f14ef18a72de884f17d0f6/users/0e022456457d41b8812058252e4a43f4/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:05:30.783248 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ffb4cc-e90f-4c3e-afa8-95b0a8d72524 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:30.783540 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ffb4cc-e90f-4c3e-afa8-95b0a8d72524 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:30.783935 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ffb4cc-e90f-4c3e-afa8-95b0a8d72524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:30.803398 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-28ffb4cc-e90f-4c3e-afa8-95b0a8d72524 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:05:30.804560 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-28ffb4cc-e90f-4c3e-afa8-95b0a8d72524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:30.836742 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:05:30.873371 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1118/2222] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:05:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:05:30.883012 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff3b06-3856-4cbe-abe0-eaeee0969529 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:30.884742 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-30ff3b06-3856-4cbe-abe0-eaeee0969529 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:30.944262 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff3b06-3856-4cbe-abe0-eaeee0969529 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:30.947996 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb4798-82e1-46fd-b95a-4cc9ff96d881 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:30.948638 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb4798-82e1-46fd-b95a-4cc9ff96d881 service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:30.948927 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb4798-82e1-46fd-b95a-4cc9ff96d881 service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:30.949205 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb4798-82e1-46fd-b95a-4cc9ff96d881 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:30.958743 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb4798-82e1-46fd-b95a-4cc9ff96d881 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:30.960174 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb4798-82e1-46fd-b95a-4cc9ff96d881 service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:30.970903 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1105/2223] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:05:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:05:35.294187 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84d85a-f66a-4b34-a57e-b6ffd8380621 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:35.294697 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84d85a-f66a-4b34-a57e-b6ffd8380621 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:35.294817 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84d85a-f66a-4b34-a57e-b6ffd8380621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:35.376563 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-9b84d85a-f66a-4b34-a57e-b6ffd8380621 None None] MFA Rules not processed for user `0e022456457d41b8812058252e4a43f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:05:35.378134 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-9b84d85a-f66a-4b34-a57e-b6ffd8380621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:35.476957 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1119/2224] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:05:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:05:35.478379 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb4798-82e1-46fd-b95a-4cc9ff96d881 service neutron] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:35.489488 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb4798-82e1-46fd-b95a-4cc9ff96d881 service neutron] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:35.494310 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cd5f4-6bb1-4036-9474-cb89b9b143c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:35.495152 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cd5f4-6bb1-4036-9474-cb89b9b143c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:35.495649 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cd5f4-6bb1-4036-9474-cb89b9b143c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:35.495939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cd5f4-6bb1-4036-9474-cb89b9b143c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:35.496655 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65cd5f4-6bb1-4036-9474-cb89b9b143c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:35.498596 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65cd5f4-6bb1-4036-9474-cb89b9b143c0 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:35.530829 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:05:35.544253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1106/2225] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:05:35 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:05:35.552570 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b84d85a-f66a-4b34-a57e-b6ffd8380621 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:35.565357 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b84d85a-f66a-4b34-a57e-b6ffd8380621 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:35.656805 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a223df-19af-4be2-8e98-8d2afa2afe94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:35.657432 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a223df-19af-4be2-8e98-8d2afa2afe94 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:35.657711 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a223df-19af-4be2-8e98-8d2afa2afe94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:35.657980 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a223df-19af-4be2-8e98-8d2afa2afe94 admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:35.658642 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a223df-19af-4be2-8e98-8d2afa2afe94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:35.660640 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a223df-19af-4be2-8e98-8d2afa2afe94 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:35.689620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1120/2226] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:05:35 2022] POST /identity/v3/users => generated 293 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:05:35.693305 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cd5f4-6bb1-4036-9474-cb89b9b143c0 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:35.704288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cd5f4-6bb1-4036-9474-cb89b9b143c0 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:35.709301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a08ae-564e-40f0-8dfb-bf760164847e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:35.709914 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a08ae-564e-40f0-8dfb-bf760164847e admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:35.710190 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a08ae-564e-40f0-8dfb-bf760164847e admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:35.710487 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a08ae-564e-40f0-8dfb-bf760164847e admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:35.711184 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a08ae-564e-40f0-8dfb-bf760164847e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:35.712829 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a08ae-564e-40f0-8dfb-bf760164847e admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:35.726278 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1107/2227] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:05:35.731027 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a223df-19af-4be2-8e98-8d2afa2afe94 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:35.741580 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a223df-19af-4be2-8e98-8d2afa2afe94 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:35.746493 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40b817-4e14-441e-8661-626cbc9e9284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:35.747250 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40b817-4e14-441e-8661-626cbc9e9284 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:35.747538 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40b817-4e14-441e-8661-626cbc9e9284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:35.747810 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40b817-4e14-441e-8661-626cbc9e9284 admin admin] PATH_INFO: `/v3/projects/ceb165f588e94eaa8f866b68dd08f3cb/users/08ad93b79675421cb798c11e1ad9473b/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:35.764816 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c40b817-4e14-441e-8661-626cbc9e9284 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceb165f588e94eaa8f866b68dd08f3cb, user_id=08ad93b79675421cb798c11e1ad9473b, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:35.767995 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c40b817-4e14-441e-8661-626cbc9e9284 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:35.782707 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1121/2228] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:05:35 2022] PUT /identity/v3/projects/ceb165f588e94eaa8f866b68dd08f3cb/users/08ad93b79675421cb798c11e1ad9473b/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:05:35.786964 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a08ae-564e-40f0-8dfb-bf760164847e admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:35.798130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a08ae-564e-40f0-8dfb-bf760164847e admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:35.869150 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d7f3b-6bf7-41ed-b895-e2722528a66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:35.869992 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d7f3b-6bf7-41ed-b895-e2722528a66a admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:35.870348 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d7f3b-6bf7-41ed-b895-e2722528a66a admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:35.870542 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d7f3b-6bf7-41ed-b895-e2722528a66a admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:35.871261 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d7f3b-6bf7-41ed-b895-e2722528a66a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:35.872907 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d7f3b-6bf7-41ed-b895-e2722528a66a admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:35.884101 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1108/2229] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:05:35.889182 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40b817-4e14-441e-8661-626cbc9e9284 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:35.902145 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40b817-4e14-441e-8661-626cbc9e9284 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:35.908102 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf4896-2f12-41f6-92e3-d6922d9227f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:35.909436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf4896-2f12-41f6-92e3-d6922d9227f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:35.909782 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf4896-2f12-41f6-92e3-d6922d9227f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:35.910118 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf4896-2f12-41f6-92e3-d6922d9227f9 admin admin] PATH_INFO: `/v3/projects/ceb165f588e94eaa8f866b68dd08f3cb/users/08ad93b79675421cb798c11e1ad9473b/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:35.912167 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf4896-2f12-41f6-92e3-d6922d9227f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceb165f588e94eaa8f866b68dd08f3cb, user_id=08ad93b79675421cb798c11e1ad9473b, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:35.914896 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf4896-2f12-41f6-92e3-d6922d9227f9 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:35.927425 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1122/2230] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:05:35 2022] PUT /identity/v3/projects/ceb165f588e94eaa8f866b68dd08f3cb/users/08ad93b79675421cb798c11e1ad9473b/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:05:41.078448 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d7f3b-6bf7-41ed-b895-e2722528a66a admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:41.080751 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-497d7f3b-6bf7-41ed-b895-e2722528a66a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:41.184730 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d7f3b-6bf7-41ed-b895-e2722528a66a admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:41.279607 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e84bb91-3395-413f-a6d1-4288a40bcd58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:41.280506 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84bb91-3395-413f-a6d1-4288a40bcd58 service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:41.280836 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a73ae-5c89-4f18-b781-683332893679 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:41.281510 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a73ae-5c89-4f18-b781-683332893679 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:41.281622 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a73ae-5c89-4f18-b781-683332893679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:41.281949 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84bb91-3395-413f-a6d1-4288a40bcd58 service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:41.282508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84bb91-3395-413f-a6d1-4288a40bcd58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:41.284652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-0e84bb91-3395-413f-a6d1-4288a40bcd58 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:41.323317 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:05:41.339098 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-7a6a73ae-5c89-4f18-b781-683332893679 None None] MFA Rules not processed for user `08ad93b79675421cb798c11e1ad9473b`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:05:41.341285 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-7a6a73ae-5c89-4f18-b781-683332893679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:41.409771 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:05:41.438471 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e84bb91-3395-413f-a6d1-4288a40bcd58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:41.440027 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e84bb91-3395-413f-a6d1-4288a40bcd58 service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:41.462185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1109/2231] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:05:41 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:05:41.490817 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1123/2232] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:05:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:05:41.497779 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e84bb91-3395-413f-a6d1-4288a40bcd58 service cinder] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:41.499668 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-0e84bb91-3395-413f-a6d1-4288a40bcd58 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:41.621695 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e84bb91-3395-413f-a6d1-4288a40bcd58 service cinder] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:41.631841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a909f-ec0d-4277-b43e-ba28e6087c49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:41.633135 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a909f-ec0d-4277-b43e-ba28e6087c49 service nova] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:41.635065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a909f-ec0d-4277-b43e-ba28e6087c49 service nova] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:41.635738 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a909f-ec0d-4277-b43e-ba28e6087c49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:41.746198 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a909f-ec0d-4277-b43e-ba28e6087c49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:41.747911 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a909f-ec0d-4277-b43e-ba28e6087c49 service nova] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:41.761583 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1110/2233] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:05:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:05:43.610674 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceccc719-00ea-4208-8ffa-2280f60b464e None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:43.611486 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceccc719-00ea-4208-8ffa-2280f60b464e None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:43.611759 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceccc719-00ea-4208-8ffa-2280f60b464e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:43.632211 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-ceccc719-00ea-4208-8ffa-2280f60b464e None None] MFA Rules not processed for user `0e022456457d41b8812058252e4a43f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:05:43.633511 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-ceccc719-00ea-4208-8ffa-2280f60b464e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:43.641778 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1124/2234] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:05:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:05:51.757171 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=60 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:05:52.051046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a909f-ec0d-4277-b43e-ba28e6087c49 service nova] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:05:52.052450 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-c62a909f-ec0d-4277-b43e-ba28e6087c49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:52.133230 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a909f-ec0d-4277-b43e-ba28e6087c49 service nova] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:05:52.138503 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f375e-8971-4b31-9314-fbe41c3b29df service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61e2c954f9a49abad9f3de17ee39f63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:05:52.139242 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f375e-8971-4b31-9314-fbe41c3b29df service placement] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:52.139536 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f375e-8971-4b31-9314-fbe41c3b29df service placement] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:52.139904 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f375e-8971-4b31-9314-fbe41c3b29df service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:52.141894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-8d0f375e-8971-4b31-9314-fbe41c3b29df service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:52.155296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f375e-8971-4b31-9314-fbe41c3b29df service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:05:52.156979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f375e-8971-4b31-9314-fbe41c3b29df service placement] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:05:52.168164 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1111/2235] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:05:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:05:52.926527 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53c38a-3585-45d9-aadc-aec526b4635c None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:05:52.926992 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53c38a-3585-45d9-aadc-aec526b4635c None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:05:52.927114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53c38a-3585-45d9-aadc-aec526b4635c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:05:52.946192 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-ea53c38a-3585-45d9-aadc-aec526b4635c None None] MFA Rules not processed for user `293b8a62f19c45c1bf5b897c7cf40edb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:05:52.947933 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-ea53c38a-3585-45d9-aadc-aec526b4635c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:05:53.018183 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1125/2236] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:05:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:06:02.162903 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=32 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:06:03.010864 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=57 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:06:26.566339 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f375e-8971-4b31-9314-fbe41c3b29df service placement] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:26.567764 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-8d0f375e-8971-4b31-9314-fbe41c3b29df service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:26.642845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f375e-8971-4b31-9314-fbe41c3b29df service placement] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:26.648138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b7dc0-3aff-471d-81c2-24010feec67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:26.648812 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b7dc0-3aff-471d-81c2-24010feec67d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:26.649089 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b7dc0-3aff-471d-81c2-24010feec67d admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:26.649360 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b7dc0-3aff-471d-81c2-24010feec67d admin admin] PATH_INFO: `/v3/users/478c4a45edad412c97680ed239931e96` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:26.650563 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5b7dc0-3aff-471d-81c2-24010feec67d admin admin] RBAC: Authorizing `identity:delete_user(user_id=478c4a45edad412c97680ed239931e96)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:26.652465 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5b7dc0-3aff-471d-81c2-24010feec67d admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:26.702868 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-7e5b7dc0-3aff-471d-81c2-24010feec67d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '478c4a45edad412c97680ed239931e96'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:26.711213 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-7e5b7dc0-3aff-471d-81c2-24010feec67d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '478c4a45edad412c97680ed239931e96'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:26.723192 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-7e5b7dc0-3aff-471d-81c2-24010feec67d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '478c4a45edad412c97680ed239931e96'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:26.723828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-7e5b7dc0-3aff-471d-81c2-24010feec67d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '478c4a45edad412c97680ed239931e96'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:26.731115 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-7e5b7dc0-3aff-471d-81c2-24010feec67d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '478c4a45edad412c97680ed239931e96'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:26.737651 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1112/2237] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:06:26 2022] DELETE /identity/v3/users/478c4a45edad412c97680ed239931e96 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:06:26.743568 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a90192-241c-428b-80c7-82ed20946b66 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:26.744914 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a90192-241c-428b-80c7-82ed20946b66 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:26.745913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a90192-241c-428b-80c7-82ed20946b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:26.767967 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-66a90192-241c-428b-80c7-82ed20946b66 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:06:26.769885 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-66a90192-241c-428b-80c7-82ed20946b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:26.834298 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1126/2238] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:06:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:06:26.845255 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b7dc0-3aff-471d-81c2-24010feec67d admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:26.846716 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-7e5b7dc0-3aff-471d-81c2-24010feec67d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:26.906374 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b7dc0-3aff-471d-81c2-24010feec67d admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:26.909984 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2171c8-f5f9-4d6f-9b6d-85ba091f764f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:26.910591 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2171c8-f5f9-4d6f-9b6d-85ba091f764f service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:26.910879 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2171c8-f5f9-4d6f-9b6d-85ba091f764f service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:26.911158 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2171c8-f5f9-4d6f-9b6d-85ba091f764f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:26.920181 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2171c8-f5f9-4d6f-9b6d-85ba091f764f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:26.921652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2171c8-f5f9-4d6f-9b6d-85ba091f764f service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:26.930979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1113/2239] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:06:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:06:27.217916 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a90192-241c-428b-80c7-82ed20946b66 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:27.219391 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-66a90192-241c-428b-80c7-82ed20946b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:27.233097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a90192-241c-428b-80c7-82ed20946b66 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:27.238198 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4cddf-2adf-4f46-a874-526984f8762a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:27.238835 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4cddf-2adf-4f46-a874-526984f8762a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:27.239110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4cddf-2adf-4f46-a874-526984f8762a admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:27.239376 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4cddf-2adf-4f46-a874-526984f8762a admin admin] PATH_INFO: `/v3/projects/87bd21f57497428cb37b5669b7679b2a` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:27.240516 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4cddf-2adf-4f46-a874-526984f8762a admin admin] RBAC: Authorizing `identity:delete_project(project_id=87bd21f57497428cb37b5669b7679b2a)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:27.243171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4cddf-2adf-4f46-a874-526984f8762a admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:27.292215 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-3eb4cddf-2adf-4f46-a874-526984f8762a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87bd21f57497428cb37b5669b7679b2a', 'request_id': 'req-3eb4cddf-2adf-4f46-a874-526984f8762a'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:27.298283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-3eb4cddf-2adf-4f46-a874-526984f8762a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87bd21f57497428cb37b5669b7679b2a', 'request_id': 'req-3eb4cddf-2adf-4f46-a874-526984f8762a'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:27.305492 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-3eb4cddf-2adf-4f46-a874-526984f8762a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87bd21f57497428cb37b5669b7679b2a', 'request_id': 'req-3eb4cddf-2adf-4f46-a874-526984f8762a'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:27.315521 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-3eb4cddf-2adf-4f46-a874-526984f8762a admin admin] The token cache is being invalidate because project 87bd21f57497428cb37b5669b7679b2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:06:27.315803 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-3eb4cddf-2adf-4f46-a874-526984f8762a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:27.316409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-3eb4cddf-2adf-4f46-a874-526984f8762a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:27.317921 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1127/2240] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:06:27 2022] DELETE /identity/v3/projects/87bd21f57497428cb37b5669b7679b2a => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:06:27.322907 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff4ff9-1eb7-4a6d-b629-e7187c524669 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:27.323205 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff4ff9-1eb7-4a6d-b629-e7187c524669 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:27.323490 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff4ff9-1eb7-4a6d-b629-e7187c524669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:27.343465 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-25ff4ff9-1eb7-4a6d-b629-e7187c524669 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:06:27.344675 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-25ff4ff9-1eb7-4a6d-b629-e7187c524669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:27.362644 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:06:27.409054 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1114/2241] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:06:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:06:27.417069 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df999d-f468-46c6-b938-26a3d0afae07 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:27.417359 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df999d-f468-46c6-b938-26a3d0afae07 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:27.417636 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df999d-f468-46c6-b938-26a3d0afae07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:27.437196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-17df999d-f468-46c6-b938-26a3d0afae07 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:06:27.438396 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-17df999d-f468-46c6-b938-26a3d0afae07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:27.446419 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1128/2242] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:06:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:06:27.451406 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff4ff9-1eb7-4a6d-b629-e7187c524669 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:27.460872 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff4ff9-1eb7-4a6d-b629-e7187c524669 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:27.466050 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dfe6ee-3e91-4ccd-b6b9-3d37e357e09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:27.466647 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfe6ee-3e91-4ccd-b6b9-3d37e357e09f admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:27.466925 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfe6ee-3e91-4ccd-b6b9-3d37e357e09f admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:27.467192 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfe6ee-3e91-4ccd-b6b9-3d37e357e09f admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:27.467874 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dfe6ee-3e91-4ccd-b6b9-3d37e357e09f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:06:27.468176 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dfe6ee-3e91-4ccd-b6b9-3d37e357e09f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:27.469794 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dfe6ee-3e91-4ccd-b6b9-3d37e357e09f admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:27.481135 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1115/2243] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:06:27.485361 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df999d-f468-46c6-b938-26a3d0afae07 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:27.494394 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df999d-f468-46c6-b938-26a3d0afae07 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:27.500129 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f559326-e51f-4a20-ba7d-ef66b540a7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:27.500841 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f559326-e51f-4a20-ba7d-ef66b540a7e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:27.501207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f559326-e51f-4a20-ba7d-ef66b540a7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:27.501541 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f559326-e51f-4a20-ba7d-ef66b540a7e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:27.502214 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f559326-e51f-4a20-ba7d-ef66b540a7e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:27.504099 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f559326-e51f-4a20-ba7d-ef66b540a7e3 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:27.535302 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1129/2244] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:06:27 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:06:27.539438 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dfe6ee-3e91-4ccd-b6b9-3d37e357e09f admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:27.550874 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dfe6ee-3e91-4ccd-b6b9-3d37e357e09f admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:27.621451 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e091e3-0c6c-4e3f-91be-cf936eab6f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:27.622106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e091e3-0c6c-4e3f-91be-cf936eab6f5d admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:27.622344 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e091e3-0c6c-4e3f-91be-cf936eab6f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:27.622621 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e091e3-0c6c-4e3f-91be-cf936eab6f5d admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:27.623345 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e091e3-0c6c-4e3f-91be-cf936eab6f5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:27.625256 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e091e3-0c6c-4e3f-91be-cf936eab6f5d admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:27.652734 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1116/2245] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:06:27 2022] POST /identity/v3/users => generated 286 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:06:27.656834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f559326-e51f-4a20-ba7d-ef66b540a7e3 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:27.668483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f559326-e51f-4a20-ba7d-ef66b540a7e3 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:27.673488 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c79c2-6d2a-4647-a151-57d6cf2276c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:27.674083 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c79c2-6d2a-4647-a151-57d6cf2276c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:27.674356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c79c2-6d2a-4647-a151-57d6cf2276c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:27.674623 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c79c2-6d2a-4647-a151-57d6cf2276c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:27.675320 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0c79c2-6d2a-4647-a151-57d6cf2276c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:27.676966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0c79c2-6d2a-4647-a151-57d6cf2276c8 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:27.688666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1130/2246] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:06:27.693043 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e091e3-0c6c-4e3f-91be-cf936eab6f5d admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:27.704178 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e091e3-0c6c-4e3f-91be-cf936eab6f5d admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:27.709473 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7794b4a-afe7-4049-a53b-649f96f357e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:27.710114 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7794b4a-afe7-4049-a53b-649f96f357e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:27.710395 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7794b4a-afe7-4049-a53b-649f96f357e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:27.710669 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7794b4a-afe7-4049-a53b-649f96f357e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:27.711371 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7794b4a-afe7-4049-a53b-649f96f357e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:27.713702 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7794b4a-afe7-4049-a53b-649f96f357e2 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:27.726580 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1117/2247] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:06:27.731001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c79c2-6d2a-4647-a151-57d6cf2276c8 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:27.741594 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c79c2-6d2a-4647-a151-57d6cf2276c8 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:27.746537 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda60ed-5dc1-47d2-a807-844895921aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:27.747255 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda60ed-5dc1-47d2-a807-844895921aa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:27.747492 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda60ed-5dc1-47d2-a807-844895921aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:27.747767 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda60ed-5dc1-47d2-a807-844895921aa2 admin admin] PATH_INFO: `/v3/projects/9b4170b01b6a46d3b8ed2bfe843e7cdb/users/c09e36fcbc3444fdaa837cf9b15a6e34/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:27.764592 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda60ed-5dc1-47d2-a807-844895921aa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b4170b01b6a46d3b8ed2bfe843e7cdb, user_id=c09e36fcbc3444fdaa837cf9b15a6e34, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:27.767569 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda60ed-5dc1-47d2-a807-844895921aa2 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:27.784074 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1131/2248] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:06:27 2022] PUT /identity/v3/projects/9b4170b01b6a46d3b8ed2bfe843e7cdb/users/c09e36fcbc3444fdaa837cf9b15a6e34/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:06:27.795378 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8e43a-dd2a-48dd-af60-368cf5e30858 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:27.795683 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8e43a-dd2a-48dd-af60-368cf5e30858 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:27.795939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8e43a-dd2a-48dd-af60-368cf5e30858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:27.829590 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-63a8e43a-dd2a-48dd-af60-368cf5e30858 None None] MFA Rules not processed for user `c09e36fcbc3444fdaa837cf9b15a6e34`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:06:27.830878 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-63a8e43a-dd2a-48dd-af60-368cf5e30858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:27.914320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1118/2249] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:06:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:06:27.926364 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda60ed-5dc1-47d2-a807-844895921aa2 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:27.933731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-eeda60ed-5dc1-47d2-a807-844895921aa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:27.966592 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:06:28.009221 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda60ed-5dc1-47d2-a807-844895921aa2 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:28.013676 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f727b1e-9473-48e0-80d4-f30c64dcbdc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:28.014286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f727b1e-9473-48e0-80d4-f30c64dcbdc2 service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:28.014562 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f727b1e-9473-48e0-80d4-f30c64dcbdc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:28.014831 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f727b1e-9473-48e0-80d4-f30c64dcbdc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:28.024186 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f727b1e-9473-48e0-80d4-f30c64dcbdc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:28.026247 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f727b1e-9473-48e0-80d4-f30c64dcbdc2 service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:28.036083 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1132/2250] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:06:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:06:28.062694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a8e43a-dd2a-48dd-af60-368cf5e30858 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:28.083134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:06:28.131247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a8e43a-dd2a-48dd-af60-368cf5e30858 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:28.136559 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcdf8cf-802e-4735-958b-4da79b875f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:28.137215 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcdf8cf-802e-4735-958b-4da79b875f19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:28.137496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcdf8cf-802e-4735-958b-4da79b875f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:28.137772 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcdf8cf-802e-4735-958b-4da79b875f19 admin admin] PATH_INFO: `/v3/users/c09e36fcbc3444fdaa837cf9b15a6e34` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:28.138946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcdf8cf-802e-4735-958b-4da79b875f19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c09e36fcbc3444fdaa837cf9b15a6e34)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:28.140993 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcdf8cf-802e-4735-958b-4da79b875f19 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:28.192909 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8fcdf8cf-802e-4735-958b-4da79b875f19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c09e36fcbc3444fdaa837cf9b15a6e34'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:28.201767 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8fcdf8cf-802e-4735-958b-4da79b875f19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c09e36fcbc3444fdaa837cf9b15a6e34'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:28.217187 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8fcdf8cf-802e-4735-958b-4da79b875f19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c09e36fcbc3444fdaa837cf9b15a6e34'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:28.217934 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8fcdf8cf-802e-4735-958b-4da79b875f19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c09e36fcbc3444fdaa837cf9b15a6e34'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:28.225517 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8fcdf8cf-802e-4735-958b-4da79b875f19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c09e36fcbc3444fdaa837cf9b15a6e34'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:28.232598 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1119/2251] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:06:28 2022] DELETE /identity/v3/users/c09e36fcbc3444fdaa837cf9b15a6e34 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:06:28.238890 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bb2a2-40f4-4aa1-9e05-be6f3228bc1e None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:28.239389 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bb2a2-40f4-4aa1-9e05-be6f3228bc1e None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:28.239909 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bb2a2-40f4-4aa1-9e05-be6f3228bc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:28.263659 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-ba2bb2a2-40f4-4aa1-9e05-be6f3228bc1e None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:06:28.264948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-ba2bb2a2-40f4-4aa1-9e05-be6f3228bc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:28.329248 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1133/2252] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:06:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:06:28.338820 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcdf8cf-802e-4735-958b-4da79b875f19 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:28.340226 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-8fcdf8cf-802e-4735-958b-4da79b875f19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:28.406503 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcdf8cf-802e-4735-958b-4da79b875f19 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:28.410244 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268f8e0-0792-4af1-9c2a-62084e39a36f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:28.410864 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268f8e0-0792-4af1-9c2a-62084e39a36f service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:28.411146 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268f8e0-0792-4af1-9c2a-62084e39a36f service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:28.411472 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268f8e0-0792-4af1-9c2a-62084e39a36f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:28.421763 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2268f8e0-0792-4af1-9c2a-62084e39a36f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:28.423235 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2268f8e0-0792-4af1-9c2a-62084e39a36f service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:28.433998 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1120/2253] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:06:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:06:28.738991 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2bb2a2-40f4-4aa1-9e05-be6f3228bc1e None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:28.740894 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-ba2bb2a2-40f4-4aa1-9e05-be6f3228bc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:28.755945 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2bb2a2-40f4-4aa1-9e05-be6f3228bc1e None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:28.775844 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87dbf3-6e17-4910-8850-ffde45bdc7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:28.777084 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87dbf3-6e17-4910-8850-ffde45bdc7a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:28.778074 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87dbf3-6e17-4910-8850-ffde45bdc7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:28.778822 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87dbf3-6e17-4910-8850-ffde45bdc7a1 admin admin] PATH_INFO: `/v3/projects/9b4170b01b6a46d3b8ed2bfe843e7cdb` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:28.781421 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c87dbf3-6e17-4910-8850-ffde45bdc7a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b4170b01b6a46d3b8ed2bfe843e7cdb)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:28.784758 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c87dbf3-6e17-4910-8850-ffde45bdc7a1 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:28.846770 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-3c87dbf3-6e17-4910-8850-ffde45bdc7a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b4170b01b6a46d3b8ed2bfe843e7cdb', 'request_id': 'req-3c87dbf3-6e17-4910-8850-ffde45bdc7a1'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:28.853340 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-3c87dbf3-6e17-4910-8850-ffde45bdc7a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b4170b01b6a46d3b8ed2bfe843e7cdb', 'request_id': 'req-3c87dbf3-6e17-4910-8850-ffde45bdc7a1'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:28.864206 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-3c87dbf3-6e17-4910-8850-ffde45bdc7a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b4170b01b6a46d3b8ed2bfe843e7cdb', 'request_id': 'req-3c87dbf3-6e17-4910-8850-ffde45bdc7a1'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:28.896515 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-3c87dbf3-6e17-4910-8850-ffde45bdc7a1 admin admin] The token cache is being invalidate because project 9b4170b01b6a46d3b8ed2bfe843e7cdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:06:28.896880 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-3c87dbf3-6e17-4910-8850-ffde45bdc7a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:28.897580 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-3c87dbf3-6e17-4910-8850-ffde45bdc7a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:28.899313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1134/2254] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:06:28 2022] DELETE /identity/v3/projects/9b4170b01b6a46d3b8ed2bfe843e7cdb => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:06:38.428504 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:06:38.879029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:06:40.044135 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268f8e0-0792-4af1-9c2a-62084e39a36f service neutron] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:40.045824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-2268f8e0-0792-4af1-9c2a-62084e39a36f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:40.062600 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268f8e0-0792-4af1-9c2a-62084e39a36f service neutron] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:40.157568 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa647c-5e3b-4f51-98b0-4b3e7f3074b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:40.158355 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa647c-5e3b-4f51-98b0-4b3e7f3074b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:40.158641 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa647c-5e3b-4f51-98b0-4b3e7f3074b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:40.158918 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa647c-5e3b-4f51-98b0-4b3e7f3074b9 admin admin] PATH_INFO: `/v3/users/d5854e86e43b493dbe722d8b9f8581c3` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:40.160116 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa647c-5e3b-4f51-98b0-4b3e7f3074b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5854e86e43b493dbe722d8b9f8581c3)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:40.162084 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa647c-5e3b-4f51-98b0-4b3e7f3074b9 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:40.231745 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-aaaa647c-5e3b-4f51-98b0-4b3e7f3074b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5854e86e43b493dbe722d8b9f8581c3'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:40.239963 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-aaaa647c-5e3b-4f51-98b0-4b3e7f3074b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5854e86e43b493dbe722d8b9f8581c3'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:40.258520 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-aaaa647c-5e3b-4f51-98b0-4b3e7f3074b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5854e86e43b493dbe722d8b9f8581c3'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:40.260491 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-aaaa647c-5e3b-4f51-98b0-4b3e7f3074b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5854e86e43b493dbe722d8b9f8581c3'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:40.269267 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-aaaa647c-5e3b-4f51-98b0-4b3e7f3074b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5854e86e43b493dbe722d8b9f8581c3'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:40.283051 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1121/2255] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:06:40 2022] DELETE /identity/v3/users/d5854e86e43b493dbe722d8b9f8581c3 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:06:40.285084 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7f8a9-4f73-4db1-87be-34e0e9795111 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:40.285552 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7f8a9-4f73-4db1-87be-34e0e9795111 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:40.286044 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7f8a9-4f73-4db1-87be-34e0e9795111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:40.310001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-5aa7f8a9-4f73-4db1-87be-34e0e9795111 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:06:40.311216 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5aa7f8a9-4f73-4db1-87be-34e0e9795111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:40.398979 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1135/2256] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:06:40.409503 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa647c-5e3b-4f51-98b0-4b3e7f3074b9 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:40.410961 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-aaaa647c-5e3b-4f51-98b0-4b3e7f3074b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:40.475104 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa647c-5e3b-4f51-98b0-4b3e7f3074b9 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:40.478856 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe862ca-044d-4e89-b217-7c18de993fd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:40.480306 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe862ca-044d-4e89-b217-7c18de993fd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:40.480671 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe862ca-044d-4e89-b217-7c18de993fd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:40.480908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe862ca-044d-4e89-b217-7c18de993fd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:40.491129 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe862ca-044d-4e89-b217-7c18de993fd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:40.492707 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe862ca-044d-4e89-b217-7c18de993fd2 service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:40.503817 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1122/2257] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:06:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:06:40.818817 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7f8a9-4f73-4db1-87be-34e0e9795111 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:40.821561 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5aa7f8a9-4f73-4db1-87be-34e0e9795111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:40.847416 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7f8a9-4f73-4db1-87be-34e0e9795111 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:40.858635 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee64139-d48d-423c-b400-485e4726cf5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:40.859368 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee64139-d48d-423c-b400-485e4726cf5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:40.859707 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee64139-d48d-423c-b400-485e4726cf5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:40.860106 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee64139-d48d-423c-b400-485e4726cf5f admin admin] PATH_INFO: `/v3/projects/04eee2a554704828ba94f8acb136d169` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:40.862192 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee64139-d48d-423c-b400-485e4726cf5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=04eee2a554704828ba94f8acb136d169)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:40.864642 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee64139-d48d-423c-b400-485e4726cf5f admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:40.933877 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-bee64139-d48d-423c-b400-485e4726cf5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04eee2a554704828ba94f8acb136d169', 'request_id': 'req-bee64139-d48d-423c-b400-485e4726cf5f'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:40.943238 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-bee64139-d48d-423c-b400-485e4726cf5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04eee2a554704828ba94f8acb136d169', 'request_id': 'req-bee64139-d48d-423c-b400-485e4726cf5f'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:40.953499 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-bee64139-d48d-423c-b400-485e4726cf5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04eee2a554704828ba94f8acb136d169', 'request_id': 'req-bee64139-d48d-423c-b400-485e4726cf5f'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:40.969406 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-bee64139-d48d-423c-b400-485e4726cf5f admin admin] The token cache is being invalidate because project 04eee2a554704828ba94f8acb136d169 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:06:40.969699 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-bee64139-d48d-423c-b400-485e4726cf5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:40.970473 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-bee64139-d48d-423c-b400-485e4726cf5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:40.972135 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1136/2258] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:06:40 2022] DELETE /identity/v3/projects/04eee2a554704828ba94f8acb136d169 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:06:40.976354 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe862ca-044d-4e89-b217-7c18de993fd2 service neutron] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:40.987182 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe862ca-044d-4e89-b217-7c18de993fd2 service neutron] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:41.005535 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:06:41.047907 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a3242-3d53-4738-be00-f313d7cd54f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:41.048558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a3242-3d53-4738-be00-f313d7cd54f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:41.048872 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a3242-3d53-4738-be00-f313d7cd54f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:41.049144 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a3242-3d53-4738-be00-f313d7cd54f4 admin admin] PATH_INFO: `/v3/users/3476c4f1de754f7ea5afb4e727f834fa` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:41.050302 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6a3242-3d53-4738-be00-f313d7cd54f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3476c4f1de754f7ea5afb4e727f834fa)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:41.052200 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6a3242-3d53-4738-be00-f313d7cd54f4 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:41.097163 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-bf6a3242-3d53-4738-be00-f313d7cd54f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3476c4f1de754f7ea5afb4e727f834fa'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:41.104221 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-bf6a3242-3d53-4738-be00-f313d7cd54f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3476c4f1de754f7ea5afb4e727f834fa'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:41.122481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-bf6a3242-3d53-4738-be00-f313d7cd54f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3476c4f1de754f7ea5afb4e727f834fa'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:41.123206 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-bf6a3242-3d53-4738-be00-f313d7cd54f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3476c4f1de754f7ea5afb4e727f834fa'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:41.130480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-bf6a3242-3d53-4738-be00-f313d7cd54f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3476c4f1de754f7ea5afb4e727f834fa'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:41.136886 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1123/2259] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:06:40 2022] DELETE /identity/v3/users/3476c4f1de754f7ea5afb4e727f834fa => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:06:41.412625 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee64139-d48d-423c-b400-485e4726cf5f admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:41.414213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-bee64139-d48d-423c-b400-485e4726cf5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:41.428045 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee64139-d48d-423c-b400-485e4726cf5f admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:41.492302 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a981fc-56bf-4589-92b2-00fbf1fb0338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:41.493104 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a981fc-56bf-4589-92b2-00fbf1fb0338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:41.494592 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a981fc-56bf-4589-92b2-00fbf1fb0338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:41.494592 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a981fc-56bf-4589-92b2-00fbf1fb0338 admin admin] PATH_INFO: `/v3/projects/1302c1f6c5b1466581bc72483e5955ec` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:41.496548 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a981fc-56bf-4589-92b2-00fbf1fb0338 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1302c1f6c5b1466581bc72483e5955ec)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:41.498886 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a981fc-56bf-4589-92b2-00fbf1fb0338 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:41.531435 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:06:41.551816 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-15a981fc-56bf-4589-92b2-00fbf1fb0338 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1302c1f6c5b1466581bc72483e5955ec', 'request_id': 'req-15a981fc-56bf-4589-92b2-00fbf1fb0338'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:41.559264 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-15a981fc-56bf-4589-92b2-00fbf1fb0338 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1302c1f6c5b1466581bc72483e5955ec', 'request_id': 'req-15a981fc-56bf-4589-92b2-00fbf1fb0338'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:41.566972 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-15a981fc-56bf-4589-92b2-00fbf1fb0338 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1302c1f6c5b1466581bc72483e5955ec', 'request_id': 'req-15a981fc-56bf-4589-92b2-00fbf1fb0338'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:41.578299 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-15a981fc-56bf-4589-92b2-00fbf1fb0338 admin admin] The token cache is being invalidate because project 1302c1f6c5b1466581bc72483e5955ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:06:41.578611 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-15a981fc-56bf-4589-92b2-00fbf1fb0338 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:41.579240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-15a981fc-56bf-4589-92b2-00fbf1fb0338 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:41.580768 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1137/2260] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:06:41 2022] DELETE /identity/v3/projects/1302c1f6c5b1466581bc72483e5955ec => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:06:41.585500 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c56b5-872b-42b7-b894-978b13fdc598 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:41.585787 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c56b5-872b-42b7-b894-978b13fdc598 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:41.586121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c56b5-872b-42b7-b894-978b13fdc598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:41.606644 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-170c56b5-872b-42b7-b894-978b13fdc598 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:06:41.607828 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-170c56b5-872b-42b7-b894-978b13fdc598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:41.666362 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1124/2261] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:06:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:06:41.674930 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3cc49-428e-42b1-b129-dbe9b4f76681 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:41.675221 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3cc49-428e-42b1-b129-dbe9b4f76681 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:41.675500 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3cc49-428e-42b1-b129-dbe9b4f76681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:41.694945 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-b7b3cc49-428e-42b1-b129-dbe9b4f76681 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:06:41.696146 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-b7b3cc49-428e-42b1-b129-dbe9b4f76681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:41.704222 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1138/2262] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:06:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:06:41.708679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c56b5-872b-42b7-b894-978b13fdc598 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:41.720261 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c56b5-872b-42b7-b894-978b13fdc598 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:41.724922 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce393b4-dd97-4aae-bc56-544b01bbfb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:41.725528 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce393b4-dd97-4aae-bc56-544b01bbfb52 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:41.725813 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce393b4-dd97-4aae-bc56-544b01bbfb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:41.726092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce393b4-dd97-4aae-bc56-544b01bbfb52 admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:41.726789 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce393b4-dd97-4aae-bc56-544b01bbfb52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:06:41.727096 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce393b4-dd97-4aae-bc56-544b01bbfb52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:41.728727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce393b4-dd97-4aae-bc56-544b01bbfb52 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:41.741948 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1125/2263] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:06:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:06:41.746386 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b3cc49-428e-42b1-b129-dbe9b4f76681 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:41.755551 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b3cc49-428e-42b1-b129-dbe9b4f76681 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:41.760675 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0584e-bdd1-46c5-9e22-424d3551b431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:41.761405 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0584e-bdd1-46c5-9e22-424d3551b431 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:41.761697 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0584e-bdd1-46c5-9e22-424d3551b431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:41.762069 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0584e-bdd1-46c5-9e22-424d3551b431 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:41.762762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0584e-bdd1-46c5-9e22-424d3551b431 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:41.764659 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0584e-bdd1-46c5-9e22-424d3551b431 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:41.794239 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1139/2264] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:06:41 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:06:41.798525 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce393b4-dd97-4aae-bc56-544b01bbfb52 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:41.808895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce393b4-dd97-4aae-bc56-544b01bbfb52 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:41.835314 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:06:41.867946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24833daa-b362-4979-8a4e-90028ff86801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:41.868552 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24833daa-b362-4979-8a4e-90028ff86801 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:41.868856 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24833daa-b362-4979-8a4e-90028ff86801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:41.869127 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24833daa-b362-4979-8a4e-90028ff86801 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:41.869774 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24833daa-b362-4979-8a4e-90028ff86801 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:41.871662 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24833daa-b362-4979-8a4e-90028ff86801 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:41.891478 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1126/2265] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:06:41 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:06:41.895576 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0584e-bdd1-46c5-9e22-424d3551b431 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:41.906449 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0584e-bdd1-46c5-9e22-424d3551b431 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:41.911051 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e03e6-5677-4bd3-8e1d-47a338077941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:41.911644 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e03e6-5677-4bd3-8e1d-47a338077941 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:41.911920 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e03e6-5677-4bd3-8e1d-47a338077941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:41.912189 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e03e6-5677-4bd3-8e1d-47a338077941 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:41.912907 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301e03e6-5677-4bd3-8e1d-47a338077941 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:41.914484 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301e03e6-5677-4bd3-8e1d-47a338077941 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:41.925847 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1140/2266] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:06:41.929629 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24833daa-b362-4979-8a4e-90028ff86801 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:41.939909 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24833daa-b362-4979-8a4e-90028ff86801 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:41.945108 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aec357-4dd9-431e-b544-aa62e442c534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:41.945717 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aec357-4dd9-431e-b544-aa62e442c534 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:41.946025 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aec357-4dd9-431e-b544-aa62e442c534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:41.946409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aec357-4dd9-431e-b544-aa62e442c534 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:41.947022 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aec357-4dd9-431e-b544-aa62e442c534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:41.948708 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aec357-4dd9-431e-b544-aa62e442c534 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:41.959981 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1127/2267] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:06:41.964552 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e03e6-5677-4bd3-8e1d-47a338077941 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:41.974592 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e03e6-5677-4bd3-8e1d-47a338077941 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:41.980065 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb1223-36f0-4678-8d22-8d8dbf1f9ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:41.980746 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb1223-36f0-4678-8d22-8d8dbf1f9ee2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:41.981024 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb1223-36f0-4678-8d22-8d8dbf1f9ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:41.981295 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb1223-36f0-4678-8d22-8d8dbf1f9ee2 admin admin] PATH_INFO: `/v3/projects/253a5db02df34de9baf31aa062da9e7c/users/f3d541abbce9420491ea25b5a9d1e10d/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:41.995346 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb1223-36f0-4678-8d22-8d8dbf1f9ee2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=253a5db02df34de9baf31aa062da9e7c, user_id=f3d541abbce9420491ea25b5a9d1e10d, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:41.998080 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb1223-36f0-4678-8d22-8d8dbf1f9ee2 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:42.011047 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1141/2268] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:06:41 2022] PUT /identity/v3/projects/253a5db02df34de9baf31aa062da9e7c/users/f3d541abbce9420491ea25b5a9d1e10d/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:06:42.021478 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078082c-2700-4a52-8ce5-ba8c434d7ed1 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:42.021763 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078082c-2700-4a52-8ce5-ba8c434d7ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:42.022035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078082c-2700-4a52-8ce5-ba8c434d7ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:42.049322 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-0078082c-2700-4a52-8ce5-ba8c434d7ed1 None None] MFA Rules not processed for user `f3d541abbce9420491ea25b5a9d1e10d`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:06:42.050507 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-0078082c-2700-4a52-8ce5-ba8c434d7ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:42.114242 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1128/2269] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:06:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:06:42.118850 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb1223-36f0-4678-8d22-8d8dbf1f9ee2 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:42.183038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb1223-36f0-4678-8d22-8d8dbf1f9ee2 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:42.187765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a244a7-b74f-4e25-b158-6108492044a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:42.188382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a244a7-b74f-4e25-b158-6108492044a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:42.188690 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a244a7-b74f-4e25-b158-6108492044a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:42.188968 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a244a7-b74f-4e25-b158-6108492044a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:42.189637 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a244a7-b74f-4e25-b158-6108492044a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:42.191516 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a244a7-b74f-4e25-b158-6108492044a4 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:42.221437 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1142/2270] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:06:42 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:06:42.225740 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078082c-2700-4a52-8ce5-ba8c434d7ed1 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:42.237050 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078082c-2700-4a52-8ce5-ba8c434d7ed1 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:42.293170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396dbe0-aff9-479f-8cdc-114d1eb60e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:42.293702 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396dbe0-aff9-479f-8cdc-114d1eb60e73 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:42.293985 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396dbe0-aff9-479f-8cdc-114d1eb60e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:42.294256 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396dbe0-aff9-479f-8cdc-114d1eb60e73 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:42.294920 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396dbe0-aff9-479f-8cdc-114d1eb60e73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:42.296847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396dbe0-aff9-479f-8cdc-114d1eb60e73 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:42.317827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1129/2271] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:06:42 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:06:42.321823 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a244a7-b74f-4e25-b158-6108492044a4 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:42.332047 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a244a7-b74f-4e25-b158-6108492044a4 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:42.336660 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23149b94-7252-446c-8be7-a95e3c13d3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:42.337261 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23149b94-7252-446c-8be7-a95e3c13d3e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:42.337543 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23149b94-7252-446c-8be7-a95e3c13d3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:42.337817 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23149b94-7252-446c-8be7-a95e3c13d3e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:42.338505 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23149b94-7252-446c-8be7-a95e3c13d3e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:42.340294 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23149b94-7252-446c-8be7-a95e3c13d3e3 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:42.350674 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1143/2272] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:06:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:06:42.355209 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396dbe0-aff9-479f-8cdc-114d1eb60e73 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:42.365180 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396dbe0-aff9-479f-8cdc-114d1eb60e73 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:42.369776 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2157a-f481-4e1d-9b86-c5b0617ceaf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:42.371188 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2157a-f481-4e1d-9b86-c5b0617ceaf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:42.371476 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2157a-f481-4e1d-9b86-c5b0617ceaf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:42.371782 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2157a-f481-4e1d-9b86-c5b0617ceaf8 admin admin] PATH_INFO: `/v3/projects/3c860f01d8c743e2bdcbab12e1ec9735/users/2f4b4578753348c3a2c256316411399c/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:42.386708 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda2157a-f481-4e1d-9b86-c5b0617ceaf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c860f01d8c743e2bdcbab12e1ec9735, user_id=2f4b4578753348c3a2c256316411399c, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:42.389519 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda2157a-f481-4e1d-9b86-c5b0617ceaf8 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:42.402788 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1130/2273] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:06:42 2022] PUT /identity/v3/projects/3c860f01d8c743e2bdcbab12e1ec9735/users/2f4b4578753348c3a2c256316411399c/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:06:42.406837 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23149b94-7252-446c-8be7-a95e3c13d3e3 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:42.417535 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23149b94-7252-446c-8be7-a95e3c13d3e3 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:42.437512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:06:42.476544 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3d7321-1f2a-4298-b196-9bf23d9d6ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:42.477176 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d7321-1f2a-4298-b196-9bf23d9d6ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:42.477457 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d7321-1f2a-4298-b196-9bf23d9d6ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:42.477746 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d7321-1f2a-4298-b196-9bf23d9d6ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:42.478415 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3d7321-1f2a-4298-b196-9bf23d9d6ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:42.480921 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3d7321-1f2a-4298-b196-9bf23d9d6ea7 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:42.490668 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1144/2274] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:06:42 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:06:42.494994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2157a-f481-4e1d-9b86-c5b0617ceaf8 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:42.500922 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:06:42.507409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2157a-f481-4e1d-9b86-c5b0617ceaf8 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:42.512733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b9426-8691-4839-a08f-cfe241b5a757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:42.513395 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b9426-8691-4839-a08f-cfe241b5a757 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:42.513677 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b9426-8691-4839-a08f-cfe241b5a757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:42.513986 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b9426-8691-4839-a08f-cfe241b5a757 admin admin] PATH_INFO: `/v3/projects/3c860f01d8c743e2bdcbab12e1ec9735/users/2f4b4578753348c3a2c256316411399c/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:42.516203 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b9426-8691-4839-a08f-cfe241b5a757 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c860f01d8c743e2bdcbab12e1ec9735, user_id=2f4b4578753348c3a2c256316411399c, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:42.518884 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b9426-8691-4839-a08f-cfe241b5a757 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:42.529692 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1131/2275] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:06:42 2022] PUT /identity/v3/projects/3c860f01d8c743e2bdcbab12e1ec9735/users/2f4b4578753348c3a2c256316411399c/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:06:42.540641 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52063cab-15a5-4b08-88f2-1bed86df280c None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:42.540962 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52063cab-15a5-4b08-88f2-1bed86df280c None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:42.541242 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52063cab-15a5-4b08-88f2-1bed86df280c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:42.568599 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-52063cab-15a5-4b08-88f2-1bed86df280c None None] MFA Rules not processed for user `2f4b4578753348c3a2c256316411399c`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:06:42.569732 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-52063cab-15a5-4b08-88f2-1bed86df280c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:42.639335 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1145/2276] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:06:42 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:06:42.650924 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b9426-8691-4839-a08f-cfe241b5a757 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:42.653274 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-548b9426-8691-4839-a08f-cfe241b5a757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:42.715638 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b9426-8691-4839-a08f-cfe241b5a757 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:42.719412 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b32c2e-5370-4d89-9ae1-289082bb2bef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:42.720060 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b32c2e-5370-4d89-9ae1-289082bb2bef service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:42.720335 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b32c2e-5370-4d89-9ae1-289082bb2bef service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:42.720640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b32c2e-5370-4d89-9ae1-289082bb2bef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:42.730915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b32c2e-5370-4d89-9ae1-289082bb2bef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:42.732335 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b32c2e-5370-4d89-9ae1-289082bb2bef service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:42.742770 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1132/2277] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:06:42 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:06:42.812690 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52063cab-15a5-4b08-88f2-1bed86df280c None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:42.874542 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52063cab-15a5-4b08-88f2-1bed86df280c None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:42.878980 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc222b12-624a-4e11-8480-7325c5ce6d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:42.879614 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc222b12-624a-4e11-8480-7325c5ce6d08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:42.879896 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc222b12-624a-4e11-8480-7325c5ce6d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:42.880173 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc222b12-624a-4e11-8480-7325c5ce6d08 admin admin] PATH_INFO: `/v3/users/f3d541abbce9420491ea25b5a9d1e10d` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:42.881375 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc222b12-624a-4e11-8480-7325c5ce6d08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3d541abbce9420491ea25b5a9d1e10d)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:42.883287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc222b12-624a-4e11-8480-7325c5ce6d08 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:42.929467 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-dc222b12-624a-4e11-8480-7325c5ce6d08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3d541abbce9420491ea25b5a9d1e10d'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:42.931074 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:06:42.937625 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-dc222b12-624a-4e11-8480-7325c5ce6d08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3d541abbce9420491ea25b5a9d1e10d'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:42.944867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-dc222b12-624a-4e11-8480-7325c5ce6d08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3d541abbce9420491ea25b5a9d1e10d'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:42.956990 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-dc222b12-624a-4e11-8480-7325c5ce6d08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3d541abbce9420491ea25b5a9d1e10d'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:42.957613 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-dc222b12-624a-4e11-8480-7325c5ce6d08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3d541abbce9420491ea25b5a9d1e10d'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:42.963376 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1146/2278] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:06:42 2022] DELETE /identity/v3/users/f3d541abbce9420491ea25b5a9d1e10d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:06:42.968711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c55e5-97ca-479c-b424-bcb05621d0b2 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:42.968994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c55e5-97ca-479c-b424-bcb05621d0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:42.969323 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c55e5-97ca-479c-b424-bcb05621d0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:42.989692 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-301c55e5-97ca-479c-b424-bcb05621d0b2 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:06:42.991539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-301c55e5-97ca-479c-b424-bcb05621d0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:43.058322 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1133/2279] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:06:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:06:43.067787 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc222b12-624a-4e11-8480-7325c5ce6d08 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:43.069200 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-dc222b12-624a-4e11-8480-7325c5ce6d08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:43.125684 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc222b12-624a-4e11-8480-7325c5ce6d08 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:43.129255 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e384ced-69e5-47b6-8b5e-95ce552f85a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:43.129869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e384ced-69e5-47b6-8b5e-95ce552f85a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:43.130148 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e384ced-69e5-47b6-8b5e-95ce552f85a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:43.130419 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e384ced-69e5-47b6-8b5e-95ce552f85a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:43.139966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e384ced-69e5-47b6-8b5e-95ce552f85a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:43.141687 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e384ced-69e5-47b6-8b5e-95ce552f85a7 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:43.151974 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1147/2280] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:06:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:06:43.421373 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c55e5-97ca-479c-b424-bcb05621d0b2 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:43.422814 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-301c55e5-97ca-479c-b424-bcb05621d0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:43.441566 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c55e5-97ca-479c-b424-bcb05621d0b2 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:43.446462 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfcd60-e737-4468-bbb0-7adaa944af3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:43.447054 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfcd60-e737-4468-bbb0-7adaa944af3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:43.447348 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfcd60-e737-4468-bbb0-7adaa944af3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:43.447595 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfcd60-e737-4468-bbb0-7adaa944af3e admin admin] PATH_INFO: `/v3/projects/253a5db02df34de9baf31aa062da9e7c` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:43.449547 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bfcd60-e737-4468-bbb0-7adaa944af3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=253a5db02df34de9baf31aa062da9e7c)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:43.451448 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bfcd60-e737-4468-bbb0-7adaa944af3e admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:43.497058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-76bfcd60-e737-4468-bbb0-7adaa944af3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '253a5db02df34de9baf31aa062da9e7c', 'request_id': 'req-76bfcd60-e737-4468-bbb0-7adaa944af3e'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:43.504355 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-76bfcd60-e737-4468-bbb0-7adaa944af3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '253a5db02df34de9baf31aa062da9e7c', 'request_id': 'req-76bfcd60-e737-4468-bbb0-7adaa944af3e'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:43.510226 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-76bfcd60-e737-4468-bbb0-7adaa944af3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '253a5db02df34de9baf31aa062da9e7c', 'request_id': 'req-76bfcd60-e737-4468-bbb0-7adaa944af3e'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:43.522197 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-76bfcd60-e737-4468-bbb0-7adaa944af3e admin admin] The token cache is being invalidate because project 253a5db02df34de9baf31aa062da9e7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:06:43.522478 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-76bfcd60-e737-4468-bbb0-7adaa944af3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:43.523094 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-76bfcd60-e737-4468-bbb0-7adaa944af3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:43.524716 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1134/2281] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:06:43 2022] DELETE /identity/v3/projects/253a5db02df34de9baf31aa062da9e7c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:06:43.528856 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e384ced-69e5-47b6-8b5e-95ce552f85a7 service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:43.538507 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e384ced-69e5-47b6-8b5e-95ce552f85a7 service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:43.596977 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d2dc3-0719-4aba-88e4-68d87c5b6a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:43.597951 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d2dc3-0719-4aba-88e4-68d87c5b6a1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:43.598261 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d2dc3-0719-4aba-88e4-68d87c5b6a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:43.598572 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d2dc3-0719-4aba-88e4-68d87c5b6a1c admin admin] PATH_INFO: `/v3/users/2f4b4578753348c3a2c256316411399c` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:43.600264 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2d2dc3-0719-4aba-88e4-68d87c5b6a1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f4b4578753348c3a2c256316411399c)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:43.602253 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2d2dc3-0719-4aba-88e4-68d87c5b6a1c admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:43.648823 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ab2d2dc3-0719-4aba-88e4-68d87c5b6a1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f4b4578753348c3a2c256316411399c'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:43.655768 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ab2d2dc3-0719-4aba-88e4-68d87c5b6a1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f4b4578753348c3a2c256316411399c'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:43.662609 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ab2d2dc3-0719-4aba-88e4-68d87c5b6a1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f4b4578753348c3a2c256316411399c'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:43.668641 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:06:43.676353 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ab2d2dc3-0719-4aba-88e4-68d87c5b6a1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f4b4578753348c3a2c256316411399c'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:43.677093 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ab2d2dc3-0719-4aba-88e4-68d87c5b6a1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f4b4578753348c3a2c256316411399c'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:43.683987 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1148/2282] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:06:43 2022] DELETE /identity/v3/users/2f4b4578753348c3a2c256316411399c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:06:43.976455 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfcd60-e737-4468-bbb0-7adaa944af3e admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:43.978001 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-76bfcd60-e737-4468-bbb0-7adaa944af3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:43.993242 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfcd60-e737-4468-bbb0-7adaa944af3e admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:44.005956 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:06:44.056211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77427f98-2ddf-422b-8c34-4036d9c879c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:44.056892 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77427f98-2ddf-422b-8c34-4036d9c879c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:44.057176 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77427f98-2ddf-422b-8c34-4036d9c879c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:44.057496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77427f98-2ddf-422b-8c34-4036d9c879c3 admin admin] PATH_INFO: `/v3/projects/3c860f01d8c743e2bdcbab12e1ec9735` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:44.058759 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77427f98-2ddf-422b-8c34-4036d9c879c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c860f01d8c743e2bdcbab12e1ec9735)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:44.060686 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77427f98-2ddf-422b-8c34-4036d9c879c3 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:44.108665 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-77427f98-2ddf-422b-8c34-4036d9c879c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c860f01d8c743e2bdcbab12e1ec9735', 'request_id': 'req-77427f98-2ddf-422b-8c34-4036d9c879c3'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:44.116288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-77427f98-2ddf-422b-8c34-4036d9c879c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c860f01d8c743e2bdcbab12e1ec9735', 'request_id': 'req-77427f98-2ddf-422b-8c34-4036d9c879c3'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:44.122289 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-77427f98-2ddf-422b-8c34-4036d9c879c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c860f01d8c743e2bdcbab12e1ec9735', 'request_id': 'req-77427f98-2ddf-422b-8c34-4036d9c879c3'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:44.133146 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-77427f98-2ddf-422b-8c34-4036d9c879c3 admin admin] The token cache is being invalidate because project 3c860f01d8c743e2bdcbab12e1ec9735 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:06:44.133428 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-77427f98-2ddf-422b-8c34-4036d9c879c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:44.134060 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-77427f98-2ddf-422b-8c34-4036d9c879c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:44.135679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1135/2283] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:06:43 2022] DELETE /identity/v3/projects/3c860f01d8c743e2bdcbab12e1ec9735 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:06:44.140552 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c2933-006c-457a-9688-1438268ffffb None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:44.141135 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c2933-006c-457a-9688-1438268ffffb None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:44.141414 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c2933-006c-457a-9688-1438268ffffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:44.162995 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-5f9c2933-006c-457a-9688-1438268ffffb None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:06:44.164198 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5f9c2933-006c-457a-9688-1438268ffffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:44.223191 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1149/2284] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:06:44.232294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07504b-a830-4f26-aca0-42224d0a7352 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:44.232598 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07504b-a830-4f26-aca0-42224d0a7352 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:44.232886 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07504b-a830-4f26-aca0-42224d0a7352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:44.252535 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-8e07504b-a830-4f26-aca0-42224d0a7352 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:06:44.253751 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-8e07504b-a830-4f26-aca0-42224d0a7352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:44.262567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1136/2285] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:06:44.266957 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c2933-006c-457a-9688-1438268ffffb None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:44.277580 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c2933-006c-457a-9688-1438268ffffb None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:44.282255 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154deaab-d9da-4b90-baf7-fc3543c30b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:44.282862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154deaab-d9da-4b90-baf7-fc3543c30b15 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:44.283138 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154deaab-d9da-4b90-baf7-fc3543c30b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:44.283405 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154deaab-d9da-4b90-baf7-fc3543c30b15 admin admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:44.284083 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154deaab-d9da-4b90-baf7-fc3543c30b15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:06:44.284382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154deaab-d9da-4b90-baf7-fc3543c30b15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:44.285985 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154deaab-d9da-4b90-baf7-fc3543c30b15 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:44.298314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1150/2286] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:06:44.302613 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07504b-a830-4f26-aca0-42224d0a7352 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:44.311760 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07504b-a830-4f26-aca0-42224d0a7352 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:44.316393 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b0371-bb33-474a-ad88-6e44d26eefe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:44.317023 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b0371-bb33-474a-ad88-6e44d26eefe3 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:44.317301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b0371-bb33-474a-ad88-6e44d26eefe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:44.317575 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b0371-bb33-474a-ad88-6e44d26eefe3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:44.318249 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b0371-bb33-474a-ad88-6e44d26eefe3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:44.320107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b0371-bb33-474a-ad88-6e44d26eefe3 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:44.347506 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1137/2287] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:06:44 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:06:44.351648 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154deaab-d9da-4b90-baf7-fc3543c30b15 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:44.361408 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154deaab-d9da-4b90-baf7-fc3543c30b15 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:44.416441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb263ea0-4044-4bc2-a86f-abcbde3a8157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:44.417087 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb263ea0-4044-4bc2-a86f-abcbde3a8157 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:44.417349 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb263ea0-4044-4bc2-a86f-abcbde3a8157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:44.417623 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb263ea0-4044-4bc2-a86f-abcbde3a8157 admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:44.418278 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb263ea0-4044-4bc2-a86f-abcbde3a8157 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:44.420163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb263ea0-4044-4bc2-a86f-abcbde3a8157 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:44.441579 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1151/2288] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:06:44 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:06:44.445674 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b0371-bb33-474a-ad88-6e44d26eefe3 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:44.456694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b0371-bb33-474a-ad88-6e44d26eefe3 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:44.461202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287337fe-b9cd-48d3-b6c8-701a0e0c200d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:44.461797 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287337fe-b9cd-48d3-b6c8-701a0e0c200d admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:44.462073 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287337fe-b9cd-48d3-b6c8-701a0e0c200d admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:44.462343 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287337fe-b9cd-48d3-b6c8-701a0e0c200d admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:44.463161 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287337fe-b9cd-48d3-b6c8-701a0e0c200d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:44.465411 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287337fe-b9cd-48d3-b6c8-701a0e0c200d admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:44.475153 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1138/2289] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:06:44.478883 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb263ea0-4044-4bc2-a86f-abcbde3a8157 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:44.489101 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb263ea0-4044-4bc2-a86f-abcbde3a8157 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:44.493686 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682567e-7df1-4c44-aec4-58ca7914435b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:44.494283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682567e-7df1-4c44-aec4-58ca7914435b admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:44.494564 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682567e-7df1-4c44-aec4-58ca7914435b admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:44.495320 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682567e-7df1-4c44-aec4-58ca7914435b admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:44.496026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5682567e-7df1-4c44-aec4-58ca7914435b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:44.497644 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5682567e-7df1-4c44-aec4-58ca7914435b admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:44.508014 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1152/2290] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:06:44.512774 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287337fe-b9cd-48d3-b6c8-701a0e0c200d admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:44.522917 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287337fe-b9cd-48d3-b6c8-701a0e0c200d admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:44.528555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364813d3-d3c1-414b-82e8-66b4151bfaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:44.529226 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364813d3-d3c1-414b-82e8-66b4151bfaa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:44.529508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364813d3-d3c1-414b-82e8-66b4151bfaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:44.529782 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364813d3-d3c1-414b-82e8-66b4151bfaa7 admin admin] PATH_INFO: `/v3/projects/b6217c51f47c430d83e820feefbf7b13/users/658cd0dff4e14c3e96100a8d94d77b25/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:44.544281 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364813d3-d3c1-414b-82e8-66b4151bfaa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6217c51f47c430d83e820feefbf7b13, user_id=658cd0dff4e14c3e96100a8d94d77b25, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:44.546933 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364813d3-d3c1-414b-82e8-66b4151bfaa7 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:44.559742 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1139/2291] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:06:44 2022] PUT /identity/v3/projects/b6217c51f47c430d83e820feefbf7b13/users/658cd0dff4e14c3e96100a8d94d77b25/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:06:44.569878 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3924dc-78ff-41bc-bec4-bbec9903f635 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:44.570325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3924dc-78ff-41bc-bec4-bbec9903f635 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:44.570432 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3924dc-78ff-41bc-bec4-bbec9903f635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:44.598892 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-be3924dc-78ff-41bc-bec4-bbec9903f635 None None] MFA Rules not processed for user `658cd0dff4e14c3e96100a8d94d77b25`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:06:44.600126 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-be3924dc-78ff-41bc-bec4-bbec9903f635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:44.641858 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:06:44.666039 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1153/2292] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:06:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:06:44.670690 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364813d3-d3c1-414b-82e8-66b4151bfaa7 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:44.719004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:06:44.741407 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364813d3-d3c1-414b-82e8-66b4151bfaa7 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:44.746134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab555e-8f25-41d6-8803-191fd473d508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:44.746739 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab555e-8f25-41d6-8803-191fd473d508 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:44.747024 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab555e-8f25-41d6-8803-191fd473d508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:44.747297 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab555e-8f25-41d6-8803-191fd473d508 admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:44.747974 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ab555e-8f25-41d6-8803-191fd473d508 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:44.749899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ab555e-8f25-41d6-8803-191fd473d508 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:44.777802 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1140/2293] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:06:44 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:06:44.781876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3924dc-78ff-41bc-bec4-bbec9903f635 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:44.795003 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3924dc-78ff-41bc-bec4-bbec9903f635 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:44.850057 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2beae4e-f528-4447-bec1-144c8c0047ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:44.850665 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2beae4e-f528-4447-bec1-144c8c0047ca admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:44.850944 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2beae4e-f528-4447-bec1-144c8c0047ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:44.851216 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2beae4e-f528-4447-bec1-144c8c0047ca admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:44.851868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2beae4e-f528-4447-bec1-144c8c0047ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:44.853800 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2beae4e-f528-4447-bec1-144c8c0047ca admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:44.874921 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1154/2294] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:06:44 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:06:44.878899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab555e-8f25-41d6-8803-191fd473d508 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:44.889511 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab555e-8f25-41d6-8803-191fd473d508 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:44.894933 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc0513-abaa-4b65-8f77-6858e6958811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:44.895476 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc0513-abaa-4b65-8f77-6858e6958811 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:44.895760 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc0513-abaa-4b65-8f77-6858e6958811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:44.896036 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc0513-abaa-4b65-8f77-6858e6958811 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:44.896827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc0513-abaa-4b65-8f77-6858e6958811 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:44.898418 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc0513-abaa-4b65-8f77-6858e6958811 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:44.909711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1141/2295] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:06:44.913848 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2beae4e-f528-4447-bec1-144c8c0047ca admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:44.924600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2beae4e-f528-4447-bec1-144c8c0047ca admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:44.929117 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0785787a-7122-4494-bab7-b536816c1aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:44.929758 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785787a-7122-4494-bab7-b536816c1aeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:44.930036 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785787a-7122-4494-bab7-b536816c1aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:44.930305 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785787a-7122-4494-bab7-b536816c1aeb admin admin] PATH_INFO: `/v3/projects/4975931567ca4ac5b5e16418b37fdf5f/users/a4a8301dd4cc413fb74947ccc0f7d33f/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:44.944969 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0785787a-7122-4494-bab7-b536816c1aeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4975931567ca4ac5b5e16418b37fdf5f, user_id=a4a8301dd4cc413fb74947ccc0f7d33f, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:44.947604 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0785787a-7122-4494-bab7-b536816c1aeb admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:44.961380 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1155/2296] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:06:44 2022] PUT /identity/v3/projects/4975931567ca4ac5b5e16418b37fdf5f/users/a4a8301dd4cc413fb74947ccc0f7d33f/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:06:44.965945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc0513-abaa-4b65-8f77-6858e6958811 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:44.976335 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc0513-abaa-4b65-8f77-6858e6958811 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:45.038060 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d46a5e-0f47-4f9e-be2e-bd04c934aa46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:45.038597 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d46a5e-0f47-4f9e-be2e-bd04c934aa46 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:45.038872 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d46a5e-0f47-4f9e-be2e-bd04c934aa46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:45.039143 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d46a5e-0f47-4f9e-be2e-bd04c934aa46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:45.039857 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d46a5e-0f47-4f9e-be2e-bd04c934aa46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:45.041513 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d46a5e-0f47-4f9e-be2e-bd04c934aa46 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:45.051911 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1142/2297] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:06:45.055869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0785787a-7122-4494-bab7-b536816c1aeb admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:45.068282 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0785787a-7122-4494-bab7-b536816c1aeb admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:45.073831 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ebb436-d37f-437e-bec8-9b2493999e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:45.074638 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebb436-d37f-437e-bec8-9b2493999e66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:45.075003 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebb436-d37f-437e-bec8-9b2493999e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:45.075278 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebb436-d37f-437e-bec8-9b2493999e66 admin admin] PATH_INFO: `/v3/projects/4975931567ca4ac5b5e16418b37fdf5f/users/a4a8301dd4cc413fb74947ccc0f7d33f/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:45.077240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ebb436-d37f-437e-bec8-9b2493999e66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4975931567ca4ac5b5e16418b37fdf5f, user_id=a4a8301dd4cc413fb74947ccc0f7d33f, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:45.079895 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ebb436-d37f-437e-bec8-9b2493999e66 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:45.094247 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1156/2298] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:06:45 2022] PUT /identity/v3/projects/4975931567ca4ac5b5e16418b37fdf5f/users/a4a8301dd4cc413fb74947ccc0f7d33f/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:06:45.106955 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9206eac-1a37-4a1e-89d0-36716adcb4aa None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:45.107426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9206eac-1a37-4a1e-89d0-36716adcb4aa None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:45.107532 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9206eac-1a37-4a1e-89d0-36716adcb4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:45.142886 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-c9206eac-1a37-4a1e-89d0-36716adcb4aa None None] MFA Rules not processed for user `a4a8301dd4cc413fb74947ccc0f7d33f`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:06:45.144227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-c9206eac-1a37-4a1e-89d0-36716adcb4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:45.212941 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1143/2299] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:06:45 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:06:45.226335 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ebb436-d37f-437e-bec8-9b2493999e66 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:45.228084 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-50ebb436-d37f-437e-bec8-9b2493999e66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:45.287163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ebb436-d37f-437e-bec8-9b2493999e66 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:45.290919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1f00b-f699-4fe9-9a41-4af074588833 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:45.291468 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1f00b-f699-4fe9-9a41-4af074588833 service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:45.291954 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1f00b-f699-4fe9-9a41-4af074588833 service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:45.292255 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1f00b-f699-4fe9-9a41-4af074588833 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:45.303945 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1f00b-f699-4fe9-9a41-4af074588833 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:45.306770 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1f00b-f699-4fe9-9a41-4af074588833 service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:45.317744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1157/2300] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:06:45 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:06:45.631161 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9206eac-1a37-4a1e-89d0-36716adcb4aa None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:45.641310 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9206eac-1a37-4a1e-89d0-36716adcb4aa None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:45.645879 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04548a2b-d60b-4011-9f33-58ee348c1d41 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a8301dd4cc413fb74947ccc0f7d33f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4975931567ca4ac5b5e16418b37fdf5f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:45.646480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04548a2b-d60b-4011-9f33-58ee348c1d41 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:45.646776 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04548a2b-d60b-4011-9f33-58ee348c1d41 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:45.647058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04548a2b-d60b-4011-9f33-58ee348c1d41 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:45.647754 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04548a2b-d60b-4011-9f33-58ee348c1d41 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:06:45.648066 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04548a2b-d60b-4011-9f33-58ee348c1d41 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:45.649803 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04548a2b-d60b-4011-9f33-58ee348c1d41 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:45.661520 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1144/2301] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:06:45.665509 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1f00b-f699-4fe9-9a41-4af074588833 service cinder] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:45.674930 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1f00b-f699-4fe9-9a41-4af074588833 service cinder] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:45.679477 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e11908-9d71-4bac-8b16-81e896aa5f04 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a8301dd4cc413fb74947ccc0f7d33f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4975931567ca4ac5b5e16418b37fdf5f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:45.680085 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e11908-9d71-4bac-8b16-81e896aa5f04 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:45.680370 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e11908-9d71-4bac-8b16-81e896aa5f04 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:45.680687 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e11908-9d71-4bac-8b16-81e896aa5f04 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:45.681361 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e11908-9d71-4bac-8b16-81e896aa5f04 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:45.683270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e11908-9d71-4bac-8b16-81e896aa5f04 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:45.711512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1158/2302] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:06:45 2022] POST /identity/v3/projects => generated 354 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:06:45.715684 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04548a2b-d60b-4011-9f33-58ee348c1d41 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:45.725784 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04548a2b-d60b-4011-9f33-58ee348c1d41 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:45.747552 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:06:45.784846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d65d06-2fad-46fb-bb26-f749b0506cf4 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a8301dd4cc413fb74947ccc0f7d33f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4975931567ca4ac5b5e16418b37fdf5f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:45.785456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d65d06-2fad-46fb-bb26-f749b0506cf4 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:45.785737 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d65d06-2fad-46fb-bb26-f749b0506cf4 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:45.786008 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d65d06-2fad-46fb-bb26-f749b0506cf4 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:45.786693 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d65d06-2fad-46fb-bb26-f749b0506cf4 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:06:45.786998 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d65d06-2fad-46fb-bb26-f749b0506cf4 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:45.788569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d65d06-2fad-46fb-bb26-f749b0506cf4 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:45.802267 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1145/2303] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:06:45.838906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e11908-9d71-4bac-8b16-81e896aa5f04 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:45.849440 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e11908-9d71-4bac-8b16-81e896aa5f04 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:45.853850 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca618d-4ff4-4b43-826f-ab57ef6bd474 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a8301dd4cc413fb74947ccc0f7d33f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4975931567ca4ac5b5e16418b37fdf5f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:45.854495 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca618d-4ff4-4b43-826f-ab57ef6bd474 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:45.854778 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca618d-4ff4-4b43-826f-ab57ef6bd474 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:45.855051 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca618d-4ff4-4b43-826f-ab57ef6bd474 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] PATH_INFO: `/v3/projects/4b8e96b1bb47418bb0bce15a3dcd1454` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:45.856945 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ca618d-4ff4-4b43-826f-ab57ef6bd474 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4b8e96b1bb47418bb0bce15a3dcd1454)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:45.858885 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ca618d-4ff4-4b43-826f-ab57ef6bd474 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:45.869827 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:06:45.906842 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-a0ca618d-4ff4-4b43-826f-ab57ef6bd474 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b8e96b1bb47418bb0bce15a3dcd1454', 'request_id': 'req-a0ca618d-4ff4-4b43-826f-ab57ef6bd474'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:45.913185 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-a0ca618d-4ff4-4b43-826f-ab57ef6bd474 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b8e96b1bb47418bb0bce15a3dcd1454', 'request_id': 'req-a0ca618d-4ff4-4b43-826f-ab57ef6bd474'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:45.921064 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-a0ca618d-4ff4-4b43-826f-ab57ef6bd474 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b8e96b1bb47418bb0bce15a3dcd1454', 'request_id': 'req-a0ca618d-4ff4-4b43-826f-ab57ef6bd474'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:45.931315 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-a0ca618d-4ff4-4b43-826f-ab57ef6bd474 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] The token cache is being invalidate because project 4b8e96b1bb47418bb0bce15a3dcd1454 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:06:45.931606 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-a0ca618d-4ff4-4b43-826f-ab57ef6bd474 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:45.932244 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-a0ca618d-4ff4-4b43-826f-ab57ef6bd474 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:45.934436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1159/2304] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:06:45 2022] DELETE /identity/v3/projects/4b8e96b1bb47418bb0bce15a3dcd1454 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:06:46.123042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d65d06-2fad-46fb-bb26-f749b0506cf4 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:46.124804 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-53d65d06-2fad-46fb-bb26-f749b0506cf4 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:46.203607 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d65d06-2fad-46fb-bb26-f749b0506cf4 tempest-VolumeQuotaClassesTest-316434927 tempest-VolumeQuotaClassesTest-316434927-project-admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:46.270700 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bcc273-da12-427e-8329-73ee3d67c141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:46.271889 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bcc273-da12-427e-8329-73ee3d67c141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:46.272179 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bcc273-da12-427e-8329-73ee3d67c141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:46.272489 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bcc273-da12-427e-8329-73ee3d67c141 admin admin] PATH_INFO: `/v3/users/658cd0dff4e14c3e96100a8d94d77b25` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:46.273729 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bcc273-da12-427e-8329-73ee3d67c141 admin admin] RBAC: Authorizing `identity:delete_user(user_id=658cd0dff4e14c3e96100a8d94d77b25)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:46.275790 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bcc273-da12-427e-8329-73ee3d67c141 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:46.329571 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-a6bcc273-da12-427e-8329-73ee3d67c141 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '658cd0dff4e14c3e96100a8d94d77b25'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:46.336698 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-a6bcc273-da12-427e-8329-73ee3d67c141 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '658cd0dff4e14c3e96100a8d94d77b25'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:46.347119 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:06:46.349226 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-a6bcc273-da12-427e-8329-73ee3d67c141 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '658cd0dff4e14c3e96100a8d94d77b25'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:46.350006 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-a6bcc273-da12-427e-8329-73ee3d67c141 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '658cd0dff4e14c3e96100a8d94d77b25'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:46.360314 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-a6bcc273-da12-427e-8329-73ee3d67c141 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '658cd0dff4e14c3e96100a8d94d77b25'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:46.366246 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1146/2305] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:06:46 2022] DELETE /identity/v3/users/658cd0dff4e14c3e96100a8d94d77b25 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:06:46.370766 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa95cb-f4cc-4577-b7c9-4da539ec1dfb None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:46.371628 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa95cb-f4cc-4577-b7c9-4da539ec1dfb None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:46.371991 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa95cb-f4cc-4577-b7c9-4da539ec1dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:46.395728 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-81fa95cb-f4cc-4577-b7c9-4da539ec1dfb None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:06:46.396979 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-81fa95cb-f4cc-4577-b7c9-4da539ec1dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:46.465959 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1160/2306] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:06:46.478267 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bcc273-da12-427e-8329-73ee3d67c141 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:46.479695 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-a6bcc273-da12-427e-8329-73ee3d67c141 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:46.540890 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bcc273-da12-427e-8329-73ee3d67c141 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:46.545088 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8456d-cf73-4b5d-96e0-bc43e8fa62c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:46.545705 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8456d-cf73-4b5d-96e0-bc43e8fa62c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:46.545989 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8456d-cf73-4b5d-96e0-bc43e8fa62c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:46.546265 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8456d-cf73-4b5d-96e0-bc43e8fa62c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:46.555875 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8456d-cf73-4b5d-96e0-bc43e8fa62c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:46.557327 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8456d-cf73-4b5d-96e0-bc43e8fa62c9 service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:46.567877 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1147/2307] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:06:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:06:46.860844 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa95cb-f4cc-4577-b7c9-4da539ec1dfb None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:46.862193 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-81fa95cb-f4cc-4577-b7c9-4da539ec1dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:46.874216 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa95cb-f4cc-4577-b7c9-4da539ec1dfb None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:46.879381 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd4f9a-6b36-4408-a797-19bd81fb4f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:46.880050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd4f9a-6b36-4408-a797-19bd81fb4f19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:46.880328 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd4f9a-6b36-4408-a797-19bd81fb4f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:46.880704 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd4f9a-6b36-4408-a797-19bd81fb4f19 admin admin] PATH_INFO: `/v3/projects/b6217c51f47c430d83e820feefbf7b13` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:46.881922 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd4f9a-6b36-4408-a797-19bd81fb4f19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6217c51f47c430d83e820feefbf7b13)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:46.883831 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd4f9a-6b36-4408-a797-19bd81fb4f19 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:46.943118 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-72bd4f9a-6b36-4408-a797-19bd81fb4f19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6217c51f47c430d83e820feefbf7b13', 'request_id': 'req-72bd4f9a-6b36-4408-a797-19bd81fb4f19'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:46.950257 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-72bd4f9a-6b36-4408-a797-19bd81fb4f19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6217c51f47c430d83e820feefbf7b13', 'request_id': 'req-72bd4f9a-6b36-4408-a797-19bd81fb4f19'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:46.958468 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-72bd4f9a-6b36-4408-a797-19bd81fb4f19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6217c51f47c430d83e820feefbf7b13', 'request_id': 'req-72bd4f9a-6b36-4408-a797-19bd81fb4f19'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:46.970756 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-72bd4f9a-6b36-4408-a797-19bd81fb4f19 admin admin] The token cache is being invalidate because project b6217c51f47c430d83e820feefbf7b13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:06:46.971051 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-72bd4f9a-6b36-4408-a797-19bd81fb4f19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:46.971724 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-72bd4f9a-6b36-4408-a797-19bd81fb4f19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:46.973314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1161/2308] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:06:46 2022] DELETE /identity/v3/projects/b6217c51f47c430d83e820feefbf7b13 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:06:46.977844 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8456d-cf73-4b5d-96e0-bc43e8fa62c9 service neutron] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:46.989239 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8456d-cf73-4b5d-96e0-bc43e8fa62c9 service neutron] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:47.051870 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164896ea-83a2-4780-9d01-9bbdbb8b167d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:47.052515 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164896ea-83a2-4780-9d01-9bbdbb8b167d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:47.052823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164896ea-83a2-4780-9d01-9bbdbb8b167d admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:47.053103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164896ea-83a2-4780-9d01-9bbdbb8b167d admin admin] PATH_INFO: `/v3/users/a4a8301dd4cc413fb74947ccc0f7d33f` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:47.054270 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164896ea-83a2-4780-9d01-9bbdbb8b167d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4a8301dd4cc413fb74947ccc0f7d33f)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:47.056181 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164896ea-83a2-4780-9d01-9bbdbb8b167d admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:47.104732 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-164896ea-83a2-4780-9d01-9bbdbb8b167d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4a8301dd4cc413fb74947ccc0f7d33f'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:47.114274 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-164896ea-83a2-4780-9d01-9bbdbb8b167d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4a8301dd4cc413fb74947ccc0f7d33f'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:47.128006 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-164896ea-83a2-4780-9d01-9bbdbb8b167d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4a8301dd4cc413fb74947ccc0f7d33f'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:47.128740 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-164896ea-83a2-4780-9d01-9bbdbb8b167d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4a8301dd4cc413fb74947ccc0f7d33f'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:47.135752 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-164896ea-83a2-4780-9d01-9bbdbb8b167d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4a8301dd4cc413fb74947ccc0f7d33f'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:47.142253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1148/2309] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:06:46 2022] DELETE /identity/v3/users/a4a8301dd4cc413fb74947ccc0f7d33f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:06:47.703749 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd4f9a-6b36-4408-a797-19bd81fb4f19 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:47.705208 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-72bd4f9a-6b36-4408-a797-19bd81fb4f19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:47.720144 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd4f9a-6b36-4408-a797-19bd81fb4f19 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:47.744448 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:06:47.784797 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca0e7fa-afa0-4498-9bc2-3454e32395d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:47.785551 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0e7fa-afa0-4498-9bc2-3454e32395d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:47.785737 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0e7fa-afa0-4498-9bc2-3454e32395d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:47.786308 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0e7fa-afa0-4498-9bc2-3454e32395d3 admin admin] PATH_INFO: `/v3/projects/4975931567ca4ac5b5e16418b37fdf5f` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:47.787464 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca0e7fa-afa0-4498-9bc2-3454e32395d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4975931567ca4ac5b5e16418b37fdf5f)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:47.789398 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca0e7fa-afa0-4498-9bc2-3454e32395d3 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:47.844548 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-dca0e7fa-afa0-4498-9bc2-3454e32395d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4975931567ca4ac5b5e16418b37fdf5f', 'request_id': 'req-dca0e7fa-afa0-4498-9bc2-3454e32395d3'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:47.851138 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-dca0e7fa-afa0-4498-9bc2-3454e32395d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4975931567ca4ac5b5e16418b37fdf5f', 'request_id': 'req-dca0e7fa-afa0-4498-9bc2-3454e32395d3'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:47.859903 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-dca0e7fa-afa0-4498-9bc2-3454e32395d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4975931567ca4ac5b5e16418b37fdf5f', 'request_id': 'req-dca0e7fa-afa0-4498-9bc2-3454e32395d3'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:47.872058 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-dca0e7fa-afa0-4498-9bc2-3454e32395d3 admin admin] The token cache is being invalidate because project 4975931567ca4ac5b5e16418b37fdf5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:06:47.872725 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-dca0e7fa-afa0-4498-9bc2-3454e32395d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:47.873144 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-dca0e7fa-afa0-4498-9bc2-3454e32395d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:06:47.874697 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1162/2310] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:06:47 2022] DELETE /identity/v3/projects/4975931567ca4ac5b5e16418b37fdf5f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:06:47.879483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee3ac8-cb41-4904-a6cc-36465bf0d3ba None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:47.879899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee3ac8-cb41-4904-a6cc-36465bf0d3ba None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:47.880175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee3ac8-cb41-4904-a6cc-36465bf0d3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:47.894631 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:06:47.901470 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-01ee3ac8-cb41-4904-a6cc-36465bf0d3ba None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:06:47.903461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-01ee3ac8-cb41-4904-a6cc-36465bf0d3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:47.967265 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1149/2311] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:06:47.974906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c6283-9e70-4904-b5e9-c742e7ab3df2 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:47.975198 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c6283-9e70-4904-b5e9-c742e7ab3df2 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:47.975511 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c6283-9e70-4904-b5e9-c742e7ab3df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:47.996622 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-387c6283-9e70-4904-b5e9-c742e7ab3df2 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:06:47.998921 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-387c6283-9e70-4904-b5e9-c742e7ab3df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:48.010974 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1163/2312] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:06:48.015142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee3ac8-cb41-4904-a6cc-36465bf0d3ba None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:48.025256 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee3ac8-cb41-4904-a6cc-36465bf0d3ba None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:48.030135 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f5860-82fd-4a0d-86da-91cd513ba765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:48.030742 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f5860-82fd-4a0d-86da-91cd513ba765 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:48.031027 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f5860-82fd-4a0d-86da-91cd513ba765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:48.031301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f5860-82fd-4a0d-86da-91cd513ba765 admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:48.031997 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f5860-82fd-4a0d-86da-91cd513ba765 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:06:48.032303 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f5860-82fd-4a0d-86da-91cd513ba765 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:48.033918 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f5860-82fd-4a0d-86da-91cd513ba765 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:48.046108 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1150/2313] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 20:06:48.050417 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c6283-9e70-4904-b5e9-c742e7ab3df2 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:48.060399 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c6283-9e70-4904-b5e9-c742e7ab3df2 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:48.065647 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861ec38-2c5e-4564-be2f-e047d42e1c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:48.066254 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861ec38-2c5e-4564-be2f-e047d42e1c75 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:48.066534 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861ec38-2c5e-4564-be2f-e047d42e1c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:48.066805 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861ec38-2c5e-4564-be2f-e047d42e1c75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:48.067478 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0861ec38-2c5e-4564-be2f-e047d42e1c75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:48.069426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0861ec38-2c5e-4564-be2f-e047d42e1c75 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:48.096961 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1164/2314] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:06:48 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:06:48.101212 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f5860-82fd-4a0d-86da-91cd513ba765 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:48.114518 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f5860-82fd-4a0d-86da-91cd513ba765 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:48.173490 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74af7fc-4f17-4060-8dc3-3a3256852c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:48.174109 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74af7fc-4f17-4060-8dc3-3a3256852c01 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:48.174386 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74af7fc-4f17-4060-8dc3-3a3256852c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:48.174655 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74af7fc-4f17-4060-8dc3-3a3256852c01 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:48.175311 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74af7fc-4f17-4060-8dc3-3a3256852c01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:48.177239 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74af7fc-4f17-4060-8dc3-3a3256852c01 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:48.201112 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1151/2315] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:06:48 2022] POST /identity/v3/users => generated 285 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:06:48.205185 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861ec38-2c5e-4564-be2f-e047d42e1c75 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:48.215965 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861ec38-2c5e-4564-be2f-e047d42e1c75 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:48.220663 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d33fb-597c-41bd-a438-ad3c1e5d7166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:48.221214 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d33fb-597c-41bd-a438-ad3c1e5d7166 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:48.221502 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d33fb-597c-41bd-a438-ad3c1e5d7166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:48.221775 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d33fb-597c-41bd-a438-ad3c1e5d7166 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:48.222473 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d33fb-597c-41bd-a438-ad3c1e5d7166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:48.224062 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d33fb-597c-41bd-a438-ad3c1e5d7166 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:48.235200 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1165/2316] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:06:48.239070 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74af7fc-4f17-4060-8dc3-3a3256852c01 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:48.250236 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74af7fc-4f17-4060-8dc3-3a3256852c01 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:48.255302 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1539d246-bf97-472e-9169-8038313eb073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:48.255923 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539d246-bf97-472e-9169-8038313eb073 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:48.256209 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539d246-bf97-472e-9169-8038313eb073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:48.256496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539d246-bf97-472e-9169-8038313eb073 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:48.257509 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1539d246-bf97-472e-9169-8038313eb073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:48.258815 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1539d246-bf97-472e-9169-8038313eb073 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:48.272805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1152/2317] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:06:48.276887 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d33fb-597c-41bd-a438-ad3c1e5d7166 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:48.297537 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d33fb-597c-41bd-a438-ad3c1e5d7166 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:48.316957 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fcdf1f-8569-498e-b63e-68d678c16984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:48.317809 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcdf1f-8569-498e-b63e-68d678c16984 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:48.318229 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcdf1f-8569-498e-b63e-68d678c16984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:48.318575 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcdf1f-8569-498e-b63e-68d678c16984 admin admin] PATH_INFO: `/v3/projects/2fee7422708346709cc0524d685031b3/users/a0dbaac800624b85ab8c07a369cf0aff/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:48.334607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fcdf1f-8569-498e-b63e-68d678c16984 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fee7422708346709cc0524d685031b3, user_id=a0dbaac800624b85ab8c07a369cf0aff, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:48.337408 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fcdf1f-8569-498e-b63e-68d678c16984 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:48.352112 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1166/2318] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:06:48 2022] PUT /identity/v3/projects/2fee7422708346709cc0524d685031b3/users/a0dbaac800624b85ab8c07a369cf0aff/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:06:48.358071 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b5d3e-ed68-421e-904e-0bade470f207 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:48.358468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b5d3e-ed68-421e-904e-0bade470f207 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:48.358805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b5d3e-ed68-421e-904e-0bade470f207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:48.378852 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-6b2b5d3e-ed68-421e-904e-0bade470f207 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:06:48.380141 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-6b2b5d3e-ed68-421e-904e-0bade470f207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:48.444453 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1153/2319] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:06:48.454171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fcdf1f-8569-498e-b63e-68d678c16984 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:48.455948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-22fcdf1f-8569-498e-b63e-68d678c16984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:48.513583 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:06:48.518822 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fcdf1f-8569-498e-b63e-68d678c16984 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:48.523297 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35bab3-32db-4188-841b-b75c4e1adfda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:48.525363 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35bab3-32db-4188-841b-b75c4e1adfda service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:48.525721 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35bab3-32db-4188-841b-b75c4e1adfda service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:48.525988 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35bab3-32db-4188-841b-b75c4e1adfda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:48.536972 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb35bab3-32db-4188-841b-b75c4e1adfda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:48.538448 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb35bab3-32db-4188-841b-b75c4e1adfda service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:48.551117 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1167/2320] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:06:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:06:49.236384 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a6e99-b599-46f6-84bd-e6b7dc014f43 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:49.237007 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a6e99-b599-46f6-84bd-e6b7dc014f43 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:49.237007 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a6e99-b599-46f6-84bd-e6b7dc014f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:49.263301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:06:49.267171 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-902a6e99-b599-46f6-84bd-e6b7dc014f43 None None] MFA Rules not processed for user `a0dbaac800624b85ab8c07a369cf0aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:06:49.269019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-902a6e99-b599-46f6-84bd-e6b7dc014f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:49.344429 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1154/2321] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:06:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:06:49.355363 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35bab3-32db-4188-841b-b75c4e1adfda service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:06:49.356888 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-bb35bab3-32db-4188-841b-b75c4e1adfda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:49.418752 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35bab3-32db-4188-841b-b75c4e1adfda service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:06:49.422101 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e3352-7652-44c2-8268-e8f5acaf8f15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:06:49.422722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e3352-7652-44c2-8268-e8f5acaf8f15 service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:49.423006 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e3352-7652-44c2-8268-e8f5acaf8f15 service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:49.423282 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e3352-7652-44c2-8268-e8f5acaf8f15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:49.431904 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980e3352-7652-44c2-8268-e8f5acaf8f15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:06:49.433927 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980e3352-7652-44c2-8268-e8f5acaf8f15 service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:06:49.442997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1168/2322] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:06:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:06:59.271691 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94556d6f-ee8d-41c9-882d-8b03dc95d783 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:06:59.272200 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94556d6f-ee8d-41c9-882d-8b03dc95d783 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:06:59.272298 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94556d6f-ee8d-41c9-882d-8b03dc95d783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:06:59.291390 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-94556d6f-ee8d-41c9-882d-8b03dc95d783 None None] MFA Rules not processed for user `a0dbaac800624b85ab8c07a369cf0aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:06:59.292569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-94556d6f-ee8d-41c9-882d-8b03dc95d783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:06:59.299578 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1155/2323] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:06:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:06:59.438898 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=30 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:07:05.967662 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41a4a0-69c1-4f1a-b8b8-f9ffdc478396 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:05.968321 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41a4a0-69c1-4f1a-b8b8-f9ffdc478396 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:05.968321 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41a4a0-69c1-4f1a-b8b8-f9ffdc478396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:05.989230 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-db41a4a0-69c1-4f1a-b8b8-f9ffdc478396 None None] MFA Rules not processed for user `ac0a233c84e1413490057c64654abe99`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:07:05.990310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-db41a4a0-69c1-4f1a-b8b8-f9ffdc478396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:07:06.060863 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1169/2324] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:07:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:07:09.286119 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=29 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:07:16.051180 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=25 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:07:17.024261 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94556d6f-ee8d-41c9-882d-8b03dc95d783 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:07:17.028312 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-94556d6f-ee8d-41c9-882d-8b03dc95d783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:07:17.055686 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94556d6f-ee8d-41c9-882d-8b03dc95d783 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:07:17.062147 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06010511-4a50-449b-9fef-14499021e2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:07:17.062798 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06010511-4a50-449b-9fef-14499021e2e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:17.063063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06010511-4a50-449b-9fef-14499021e2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:17.063336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06010511-4a50-449b-9fef-14499021e2e4 admin admin] PATH_INFO: `/v3/users/2d8b9ca1937c41af8c34176f589e44a8` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:17.064499 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06010511-4a50-449b-9fef-14499021e2e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d8b9ca1937c41af8c34176f589e44a8)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:07:17.066580 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06010511-4a50-449b-9fef-14499021e2e4 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:07:17.160938 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-06010511-4a50-449b-9fef-14499021e2e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d8b9ca1937c41af8c34176f589e44a8'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:07:17.173012 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-06010511-4a50-449b-9fef-14499021e2e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d8b9ca1937c41af8c34176f589e44a8'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:07:17.189506 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-06010511-4a50-449b-9fef-14499021e2e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d8b9ca1937c41af8c34176f589e44a8'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:07:17.190301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-06010511-4a50-449b-9fef-14499021e2e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d8b9ca1937c41af8c34176f589e44a8'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:07:17.198611 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-06010511-4a50-449b-9fef-14499021e2e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d8b9ca1937c41af8c34176f589e44a8'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:07:17.206705 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1156/2325] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:07:17 2022] DELETE /identity/v3/users/2d8b9ca1937c41af8c34176f589e44a8 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:07:17.211217 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2bf82-67bf-4ffd-959f-139754bf1762 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:17.211758 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2bf82-67bf-4ffd-959f-139754bf1762 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:17.211758 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2bf82-67bf-4ffd-959f-139754bf1762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:17.240294 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-aba2bf82-67bf-4ffd-959f-139754bf1762 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:07:17.241856 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-aba2bf82-67bf-4ffd-959f-139754bf1762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:07:17.324546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1170/2326] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:07:17.331564 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06010511-4a50-449b-9fef-14499021e2e4 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:07:17.333144 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-06010511-4a50-449b-9fef-14499021e2e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:07:17.415720 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06010511-4a50-449b-9fef-14499021e2e4 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:07:17.421050 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653cdd7c-5ba8-4517-adef-e1c84a82a24e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:07:17.421848 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653cdd7c-5ba8-4517-adef-e1c84a82a24e service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:17.422496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653cdd7c-5ba8-4517-adef-e1c84a82a24e service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:17.422790 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653cdd7c-5ba8-4517-adef-e1c84a82a24e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:17.437936 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653cdd7c-5ba8-4517-adef-e1c84a82a24e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:07:17.439392 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653cdd7c-5ba8-4517-adef-e1c84a82a24e service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:07:17.451312 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1157/2327] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:07:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:07:18.147576 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2bf82-67bf-4ffd-959f-139754bf1762 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:07:18.149018 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-aba2bf82-67bf-4ffd-959f-139754bf1762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:07:18.161577 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2bf82-67bf-4ffd-959f-139754bf1762 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:07:18.166491 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d16b1-c948-41d8-9ec6-f22b2c200dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:07:18.167277 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d16b1-c948-41d8-9ec6-f22b2c200dc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:18.167637 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d16b1-c948-41d8-9ec6-f22b2c200dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:18.168095 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d16b1-c948-41d8-9ec6-f22b2c200dc9 admin admin] PATH_INFO: `/v3/projects/08cb43e2c9c84cffa058389e8fa59dd9` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:18.169901 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498d16b1-c948-41d8-9ec6-f22b2c200dc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08cb43e2c9c84cffa058389e8fa59dd9)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:07:18.171928 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498d16b1-c948-41d8-9ec6-f22b2c200dc9 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:07:18.223066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-498d16b1-c948-41d8-9ec6-f22b2c200dc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08cb43e2c9c84cffa058389e8fa59dd9', 'request_id': 'req-498d16b1-c948-41d8-9ec6-f22b2c200dc9'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:07:18.229329 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-498d16b1-c948-41d8-9ec6-f22b2c200dc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08cb43e2c9c84cffa058389e8fa59dd9', 'request_id': 'req-498d16b1-c948-41d8-9ec6-f22b2c200dc9'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:07:18.237324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-498d16b1-c948-41d8-9ec6-f22b2c200dc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08cb43e2c9c84cffa058389e8fa59dd9', 'request_id': 'req-498d16b1-c948-41d8-9ec6-f22b2c200dc9'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:07:18.248728 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-498d16b1-c948-41d8-9ec6-f22b2c200dc9 admin admin] The token cache is being invalidate because project 08cb43e2c9c84cffa058389e8fa59dd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:07:18.249019 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-498d16b1-c948-41d8-9ec6-f22b2c200dc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:07:18.249677 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-498d16b1-c948-41d8-9ec6-f22b2c200dc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:07:18.251183 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1171/2328] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:07:18 2022] DELETE /identity/v3/projects/08cb43e2c9c84cffa058389e8fa59dd9 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:07:18.256095 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b114e-581c-4133-b06f-3f2805a6a4e3 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:18.256554 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b114e-581c-4133-b06f-3f2805a6a4e3 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:18.256694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b114e-581c-4133-b06f-3f2805a6a4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:18.276764 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-ba1b114e-581c-4133-b06f-3f2805a6a4e3 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:07:18.278537 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-ba1b114e-581c-4133-b06f-3f2805a6a4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:07:18.339309 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1158/2329] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:07:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:07:18.347014 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c30f68-f15a-4d33-9674-02e269f987ba None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:18.347302 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c30f68-f15a-4d33-9674-02e269f987ba None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:18.347580 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c30f68-f15a-4d33-9674-02e269f987ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:18.367759 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-a3c30f68-f15a-4d33-9674-02e269f987ba None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:07:18.369096 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-a3c30f68-f15a-4d33-9674-02e269f987ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:07:18.378493 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1172/2330] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:07:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:07:18.382758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b114e-581c-4133-b06f-3f2805a6a4e3 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:07:18.393768 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b114e-581c-4133-b06f-3f2805a6a4e3 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:07:18.398336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255342c9-cd4d-4561-9423-c9b827a73d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:07:18.398947 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255342c9-cd4d-4561-9423-c9b827a73d6d admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:18.399236 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255342c9-cd4d-4561-9423-c9b827a73d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:18.399509 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255342c9-cd4d-4561-9423-c9b827a73d6d admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:18.400201 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255342c9-cd4d-4561-9423-c9b827a73d6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:07:18.400506 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255342c9-cd4d-4561-9423-c9b827a73d6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:07:18.402127 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255342c9-cd4d-4561-9423-c9b827a73d6d admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:07:18.414045 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1159/2331] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:07:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:07:18.418465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c30f68-f15a-4d33-9674-02e269f987ba None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:07:18.428068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c30f68-f15a-4d33-9674-02e269f987ba None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:07:18.432821 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbb9ce-a05f-4aa4-9b10-bac8860f4309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:07:18.434222 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbb9ce-a05f-4aa4-9b10-bac8860f4309 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:18.434519 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbb9ce-a05f-4aa4-9b10-bac8860f4309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:18.434519 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbb9ce-a05f-4aa4-9b10-bac8860f4309 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:18.435337 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bbb9ce-a05f-4aa4-9b10-bac8860f4309 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:07:18.437297 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bbb9ce-a05f-4aa4-9b10-bac8860f4309 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:07:18.467023 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1173/2332] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:07:18 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:07:18.471180 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255342c9-cd4d-4561-9423-c9b827a73d6d admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:07:18.476471 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:07:18.482003 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255342c9-cd4d-4561-9423-c9b827a73d6d admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:07:18.552984 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c027287-1a2f-4301-aaa6-d49bc88e77e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:07:18.553594 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c027287-1a2f-4301-aaa6-d49bc88e77e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:18.553872 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c027287-1a2f-4301-aaa6-d49bc88e77e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:18.554143 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c027287-1a2f-4301-aaa6-d49bc88e77e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:18.554806 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c027287-1a2f-4301-aaa6-d49bc88e77e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:07:18.556716 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c027287-1a2f-4301-aaa6-d49bc88e77e3 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:07:18.580864 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1160/2333] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:07:18 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:07:18.584815 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbb9ce-a05f-4aa4-9b10-bac8860f4309 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:07:18.595201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbb9ce-a05f-4aa4-9b10-bac8860f4309 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:07:18.599857 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc013220-aa84-440e-b54d-0501990d280c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:07:18.600467 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc013220-aa84-440e-b54d-0501990d280c admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:18.600776 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc013220-aa84-440e-b54d-0501990d280c admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:18.601059 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc013220-aa84-440e-b54d-0501990d280c admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:18.601755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc013220-aa84-440e-b54d-0501990d280c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:07:18.603348 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc013220-aa84-440e-b54d-0501990d280c admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:07:18.614456 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1174/2334] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:07:18.618447 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c027287-1a2f-4301-aaa6-d49bc88e77e3 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:07:18.629307 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c027287-1a2f-4301-aaa6-d49bc88e77e3 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:07:18.634117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db704dd4-0c00-494c-9abc-bd445ad25543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:07:18.634755 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db704dd4-0c00-494c-9abc-bd445ad25543 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:18.635036 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db704dd4-0c00-494c-9abc-bd445ad25543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:18.635312 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db704dd4-0c00-494c-9abc-bd445ad25543 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:18.636059 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db704dd4-0c00-494c-9abc-bd445ad25543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:07:18.638433 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db704dd4-0c00-494c-9abc-bd445ad25543 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:07:18.649696 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1161/2335] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:07:18.653959 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc013220-aa84-440e-b54d-0501990d280c admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:07:18.664175 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc013220-aa84-440e-b54d-0501990d280c admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:07:18.669541 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf27eb3-8aea-4072-b3aa-4e25fb578b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:07:18.670207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf27eb3-8aea-4072-b3aa-4e25fb578b7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:18.670486 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf27eb3-8aea-4072-b3aa-4e25fb578b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:18.670756 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf27eb3-8aea-4072-b3aa-4e25fb578b7e admin admin] PATH_INFO: `/v3/projects/fe26e033d1844e28849ed4de857a7958/users/368838695ea74570b7e30c17a211b82b/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:18.686260 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf27eb3-8aea-4072-b3aa-4e25fb578b7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe26e033d1844e28849ed4de857a7958, user_id=368838695ea74570b7e30c17a211b82b, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:07:18.688925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf27eb3-8aea-4072-b3aa-4e25fb578b7e admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:07:18.702600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1175/2336] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:07:18 2022] PUT /identity/v3/projects/fe26e033d1844e28849ed4de857a7958/users/368838695ea74570b7e30c17a211b82b/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:07:18.707742 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f5beb-74e8-4a91-b2f6-f580f1fb99f2 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:18.708122 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f5beb-74e8-4a91-b2f6-f580f1fb99f2 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:18.708404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f5beb-74e8-4a91-b2f6-f580f1fb99f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:18.727772 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-ad8f5beb-74e8-4a91-b2f6-f580f1fb99f2 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:07:18.729005 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-ad8f5beb-74e8-4a91-b2f6-f580f1fb99f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:07:18.789478 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1162/2337] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:07:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:07:18.800136 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf27eb3-8aea-4072-b3aa-4e25fb578b7e admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:07:18.801850 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-ebf27eb3-8aea-4072-b3aa-4e25fb578b7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:07:18.823676 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:07:18.861578 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf27eb3-8aea-4072-b3aa-4e25fb578b7e admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:07:18.865364 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1609b59-bc98-4f5b-a310-eb9314613d66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:07:18.865975 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1609b59-bc98-4f5b-a310-eb9314613d66 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:18.866252 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1609b59-bc98-4f5b-a310-eb9314613d66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:18.866549 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1609b59-bc98-4f5b-a310-eb9314613d66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:18.876310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1609b59-bc98-4f5b-a310-eb9314613d66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:07:18.877788 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1609b59-bc98-4f5b-a310-eb9314613d66 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:07:18.887217 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1176/2338] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:07:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:07:23.770433 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb294e70-238c-45b0-87f2-fd2374f190e9 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:23.770895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb294e70-238c-45b0-87f2-fd2374f190e9 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:23.771002 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb294e70-238c-45b0-87f2-fd2374f190e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:23.826863 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-bb294e70-238c-45b0-87f2-fd2374f190e9 None None] MFA Rules not processed for user `368838695ea74570b7e30c17a211b82b`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:07:23.828075 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-bb294e70-238c-45b0-87f2-fd2374f190e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:07:23.897774 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1163/2339] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:07:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:07:23.902953 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1609b59-bc98-4f5b-a310-eb9314613d66 service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:07:23.912935 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1609b59-bc98-4f5b-a310-eb9314613d66 service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:07:23.918405 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151956b4-022f-41a7-9c1b-98d6abacf863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:07:23.919082 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151956b4-022f-41a7-9c1b-98d6abacf863 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:23.919372 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151956b4-022f-41a7-9c1b-98d6abacf863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:23.919649 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151956b4-022f-41a7-9c1b-98d6abacf863 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:23.920333 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151956b4-022f-41a7-9c1b-98d6abacf863 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:07:23.924960 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151956b4-022f-41a7-9c1b-98d6abacf863 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:07:23.956108 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1177/2340] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:07:23 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:07:23.960305 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb294e70-238c-45b0-87f2-fd2374f190e9 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:07:23.971827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb294e70-238c-45b0-87f2-fd2374f190e9 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:07:23.981245 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:07:24.036869 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3160b-98ba-462e-9b14-d43c1545037d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:07:24.037425 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3160b-98ba-462e-9b14-d43c1545037d admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:24.037685 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3160b-98ba-462e-9b14-d43c1545037d admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:24.037954 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3160b-98ba-462e-9b14-d43c1545037d admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:24.038628 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f3160b-98ba-462e-9b14-d43c1545037d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:07:24.040531 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f3160b-98ba-462e-9b14-d43c1545037d admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:07:24.063944 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1164/2341] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:07:23 2022] POST /identity/v3/users => generated 299 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:07:24.067764 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151956b4-022f-41a7-9c1b-98d6abacf863 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:07:24.079505 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151956b4-022f-41a7-9c1b-98d6abacf863 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:07:24.084539 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f3b04-e237-434d-b65f-23c82efb0a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:07:24.085285 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f3b04-e237-434d-b65f-23c82efb0a51 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:24.085635 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f3b04-e237-434d-b65f-23c82efb0a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:24.086013 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f3b04-e237-434d-b65f-23c82efb0a51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:24.086783 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f3b04-e237-434d-b65f-23c82efb0a51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:07:24.088461 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f3b04-e237-434d-b65f-23c82efb0a51 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:07:24.100943 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1178/2342] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:07:24.105204 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3160b-98ba-462e-9b14-d43c1545037d admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:07:24.115771 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3160b-98ba-462e-9b14-d43c1545037d admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:07:24.123869 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54028d14-f253-4ae1-8ea4-29e2ae35f823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:07:24.124556 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54028d14-f253-4ae1-8ea4-29e2ae35f823 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:24.124866 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54028d14-f253-4ae1-8ea4-29e2ae35f823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:24.125140 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54028d14-f253-4ae1-8ea4-29e2ae35f823 admin admin] PATH_INFO: `/v3/projects/f44e23eb59d04a8d96ac26782c467d99/users/4955744df809448f814df2b6777cb9cb/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:24.142780 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54028d14-f253-4ae1-8ea4-29e2ae35f823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f44e23eb59d04a8d96ac26782c467d99, user_id=4955744df809448f814df2b6777cb9cb, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:07:24.145596 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54028d14-f253-4ae1-8ea4-29e2ae35f823 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:07:24.161467 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1165/2343] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:07:24 2022] PUT /identity/v3/projects/f44e23eb59d04a8d96ac26782c467d99/users/4955744df809448f814df2b6777cb9cb/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:07:24.165052 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f3b04-e237-434d-b65f-23c82efb0a51 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:07:24.177444 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f3b04-e237-434d-b65f-23c82efb0a51 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:07:24.257687 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f42f3d-7b6d-4ad8-b70d-bac09ed0b913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:07:24.258419 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f42f3d-7b6d-4ad8-b70d-bac09ed0b913 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:24.258704 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f42f3d-7b6d-4ad8-b70d-bac09ed0b913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:24.258979 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f42f3d-7b6d-4ad8-b70d-bac09ed0b913 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:24.259672 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f42f3d-7b6d-4ad8-b70d-bac09ed0b913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:07:24.261449 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f42f3d-7b6d-4ad8-b70d-bac09ed0b913 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:07:24.277528 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1179/2344] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:07:24.278261 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54028d14-f253-4ae1-8ea4-29e2ae35f823 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:07:24.288931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54028d14-f253-4ae1-8ea4-29e2ae35f823 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:07:24.294295 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5579df-e19e-4c83-9e6e-f3f2c9aa0201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:07:24.295105 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5579df-e19e-4c83-9e6e-f3f2c9aa0201 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:24.295444 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5579df-e19e-4c83-9e6e-f3f2c9aa0201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:24.295782 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5579df-e19e-4c83-9e6e-f3f2c9aa0201 admin admin] PATH_INFO: `/v3/projects/f44e23eb59d04a8d96ac26782c467d99/users/4955744df809448f814df2b6777cb9cb/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:24.297826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5579df-e19e-4c83-9e6e-f3f2c9aa0201 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f44e23eb59d04a8d96ac26782c467d99, user_id=4955744df809448f814df2b6777cb9cb, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:07:24.300732 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5579df-e19e-4c83-9e6e-f3f2c9aa0201 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:07:24.312917 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1166/2345] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:07:24 2022] PUT /identity/v3/projects/f44e23eb59d04a8d96ac26782c467d99/users/4955744df809448f814df2b6777cb9cb/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:07:29.421170 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec292d-3a44-4320-800e-808aac9741c6 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:29.421749 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec292d-3a44-4320-800e-808aac9741c6 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:29.421917 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec292d-3a44-4320-800e-808aac9741c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:29.454288 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-5eec292d-3a44-4320-800e-808aac9741c6 None None] MFA Rules not processed for user `4955744df809448f814df2b6777cb9cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:07:29.455528 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5eec292d-3a44-4320-800e-808aac9741c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:07:29.531806 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1180/2346] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:07:29 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:07:29.539380 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5579df-e19e-4c83-9e6e-f3f2c9aa0201 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:07:29.541162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-ed5579df-e19e-4c83-9e6e-f3f2c9aa0201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:07:29.606336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5579df-e19e-4c83-9e6e-f3f2c9aa0201 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:07:29.610140 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07624e6a-d852-4559-94da-e1e271d75771 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe69b398c1f4e18b625a63ae40b01e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:07:29.610759 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07624e6a-d852-4559-94da-e1e271d75771 service glance] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:29.611043 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07624e6a-d852-4559-94da-e1e271d75771 service glance] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:29.611318 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07624e6a-d852-4559-94da-e1e271d75771 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:29.690178 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07624e6a-d852-4559-94da-e1e271d75771 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:07:29.693526 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07624e6a-d852-4559-94da-e1e271d75771 service glance] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:07:29.705892 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1167/2347] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:07:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:07:29.849736 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ddfb1c-ae89-41b4-9bda-4078fc716b7c None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:29.850148 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ddfb1c-ae89-41b4-9bda-4078fc716b7c None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:29.850512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ddfb1c-ae89-41b4-9bda-4078fc716b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:29.872450 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-37ddfb1c-ae89-41b4-9bda-4078fc716b7c None None] MFA Rules not processed for user `3488a7d2258242c0859a105c1d162767`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:07:29.873700 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-37ddfb1c-ae89-41b4-9bda-4078fc716b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:07:29.880019 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:07:29.959762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1181/2348] 10.222.0.39 () {58 vars in 1009 bytes} [Fri Aug 5 20:07:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:07:30.014463 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07624e6a-d852-4559-94da-e1e271d75771 service glance] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:07:30.016412 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-07624e6a-d852-4559-94da-e1e271d75771 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:07:30.050053 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:07:30.117585 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07624e6a-d852-4559-94da-e1e271d75771 service glance] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:07:30.125775 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1bd653-8a0a-4e8f-951a-8cfc8a2acbc2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91d3d43cb0bd4ca3b8e8f10e29cc421c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:07:30.127117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bd653-8a0a-4e8f-951a-8cfc8a2acbc2 service swift] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:30.127117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bd653-8a0a-4e8f-951a-8cfc8a2acbc2 service swift] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:30.127117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bd653-8a0a-4e8f-951a-8cfc8a2acbc2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:30.136397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1bd653-8a0a-4e8f-951a-8cfc8a2acbc2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:07:30.138136 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1bd653-8a0a-4e8f-951a-8cfc8a2acbc2 service swift] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:07:30.170016 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1168/2349] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Aug 5 20:07:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 20:07:33.571007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1fc0f0-b66b-4129-9ade-e6418a99a32c None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:33.571567 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1fc0f0-b66b-4129-9ade-e6418a99a32c None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:33.571755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1fc0f0-b66b-4129-9ade-e6418a99a32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:33.594576 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-3f1fc0f0-b66b-4129-9ade-e6418a99a32c None None] MFA Rules not processed for user `a0dbaac800624b85ab8c07a369cf0aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:07:33.595851 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-3f1fc0f0-b66b-4129-9ade-e6418a99a32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:07:33.670927 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1182/2350] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:07:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:07:35.403787 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1bd653-8a0a-4e8f-951a-8cfc8a2acbc2 service swift] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:07:35.489848 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1bd653-8a0a-4e8f-951a-8cfc8a2acbc2 service swift] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:07:35.493756 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c8812-39fe-44be-b2b7-d3e9e6bcdd87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:07:35.494467 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c8812-39fe-44be-b2b7-d3e9e6bcdd87 service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:35.494772 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c8812-39fe-44be-b2b7-d3e9e6bcdd87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:35.495923 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c8812-39fe-44be-b2b7-d3e9e6bcdd87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:35.498054 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-db6c8812-39fe-44be-b2b7-d3e9e6bcdd87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:07:35.630386 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c8812-39fe-44be-b2b7-d3e9e6bcdd87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:07:35.632470 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c8812-39fe-44be-b2b7-d3e9e6bcdd87 service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:07:35.647506 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1169/2351] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:07:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:07:36.311194 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1fc0f0-b66b-4129-9ade-e6418a99a32c None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:07:36.314696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-3f1fc0f0-b66b-4129-9ade-e6418a99a32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:07:36.339922 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1fc0f0-b66b-4129-9ade-e6418a99a32c None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:07:36.356733 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aad534-cc89-4c5e-9f04-f48574029906 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:07:36.357511 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aad534-cc89-4c5e-9f04-f48574029906 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:36.357803 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aad534-cc89-4c5e-9f04-f48574029906 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:36.358080 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aad534-cc89-4c5e-9f04-f48574029906 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:36.362674 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-b9aad534-cc89-4c5e-9f04-f48574029906 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:07:36.484492 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aad534-cc89-4c5e-9f04-f48574029906 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:07:36.486381 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aad534-cc89-4c5e-9f04-f48574029906 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:07:36.505858 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1183/2352] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:07:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:07:38.184227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c8812-39fe-44be-b2b7-d3e9e6bcdd87 service neutron] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:07:38.197532 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c8812-39fe-44be-b2b7-d3e9e6bcdd87 service neutron] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:07:38.214701 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822566e-2f3d-4a15-b8f8-417621e6ac44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:07:38.215848 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822566e-2f3d-4a15-b8f8-417621e6ac44 service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:38.215988 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822566e-2f3d-4a15-b8f8-417621e6ac44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:38.216143 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822566e-2f3d-4a15-b8f8-417621e6ac44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:38.220559 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-0822566e-2f3d-4a15-b8f8-417621e6ac44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:07:38.259152 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0822566e-2f3d-4a15-b8f8-417621e6ac44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:07:38.259152 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0822566e-2f3d-4a15-b8f8-417621e6ac44 service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:07:38.279569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1170/2353] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:07:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:07:41.653753 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88c5d9-2019-4afd-937d-88837240484c None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:41.654257 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88c5d9-2019-4afd-937d-88837240484c None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:41.654368 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88c5d9-2019-4afd-937d-88837240484c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:41.673422 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-da88c5d9-2019-4afd-937d-88837240484c None None] MFA Rules not processed for user `368838695ea74570b7e30c17a211b82b`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:07:41.674617 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-da88c5d9-2019-4afd-937d-88837240484c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:07:41.681431 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1184/2354] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:07:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:07:45.302010 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d30f74-2fe1-453a-b328-2851a52c56ff None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:45.302561 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d30f74-2fe1-453a-b328-2851a52c56ff None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:45.302667 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d30f74-2fe1-453a-b328-2851a52c56ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:45.324364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-b5d30f74-2fe1-453a-b328-2851a52c56ff None None] MFA Rules not processed for user `3488a7d2258242c0859a105c1d162767`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:07:45.325757 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-b5d30f74-2fe1-453a-b328-2851a52c56ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:07:45.332958 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1171/2355] 10.222.0.39 () {58 vars in 1009 bytes} [Fri Aug 5 20:07:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:07:45.346804 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da88c5d9-2019-4afd-937d-88837240484c None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:07:45.359820 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da88c5d9-2019-4afd-937d-88837240484c None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:07:45.365532 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0bf2d5-50e8-4f56-aac3-97f37643c42f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91d3d43cb0bd4ca3b8e8f10e29cc421c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:07:45.366063 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0bf2d5-50e8-4f56-aac3-97f37643c42f service swift] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:45.366348 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0bf2d5-50e8-4f56-aac3-97f37643c42f service swift] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:45.366670 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0bf2d5-50e8-4f56-aac3-97f37643c42f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:45.378670 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0bf2d5-50e8-4f56-aac3-97f37643c42f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:07:45.381039 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0bf2d5-50e8-4f56-aac3-97f37643c42f service swift] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:07:45.395007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1185/2356] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Aug 5 20:07:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 20:07:48.344170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d30f74-2fe1-453a-b328-2851a52c56ff None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:07:48.345745 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-b5d30f74-2fe1-453a-b328-2851a52c56ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:07:48.361875 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d30f74-2fe1-453a-b328-2851a52c56ff None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:07:48.368467 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a26d2-a605-450d-90fd-f8df16fccd18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:07:48.369712 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a26d2-a605-450d-90fd-f8df16fccd18 service nova] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:07:48.370175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a26d2-a605-450d-90fd-f8df16fccd18 service nova] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:07:48.370448 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a26d2-a605-450d-90fd-f8df16fccd18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:07:48.372160 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-554a26d2-a605-450d-90fd-f8df16fccd18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:07:48.401656 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:07:48.435159 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a26d2-a605-450d-90fd-f8df16fccd18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:07:48.436766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a26d2-a605-450d-90fd-f8df16fccd18 service nova] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:07:48.445684 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1172/2357] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:07:48 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:07:55.388071 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=84 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:07:58.441455 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=15 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:08:06.393104 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0bf2d5-50e8-4f56-aac3-97f37643c42f service swift] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:08:06.402680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0bf2d5-50e8-4f56-aac3-97f37643c42f service swift] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:08:06.408026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5003b1e-0999-4747-837c-34000a8ea419 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:08:06.408666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5003b1e-0999-4747-837c-34000a8ea419 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:08:06.408947 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5003b1e-0999-4747-837c-34000a8ea419 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:08:06.409509 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5003b1e-0999-4747-837c-34000a8ea419 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:08:06.411763 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e5003b1e-0999-4747-837c-34000a8ea419 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:08:06.423475 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5003b1e-0999-4747-837c-34000a8ea419 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:08:06.424946 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5003b1e-0999-4747-837c-34000a8ea419 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:08:06.433298 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1186/2358] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:08:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:08:16.429735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=6 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:08:20.364144 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a26d2-a605-450d-90fd-f8df16fccd18 service nova] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:08:20.365769 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-554a26d2-a605-450d-90fd-f8df16fccd18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:08:20.380231 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a26d2-a605-450d-90fd-f8df16fccd18 service nova] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:08:20.385717 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1cbc63-3907-45e7-a2ed-0eb0b7d8541d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:08:20.386319 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cbc63-3907-45e7-a2ed-0eb0b7d8541d service nova] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:08:20.386595 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cbc63-3907-45e7-a2ed-0eb0b7d8541d service nova] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:08:20.386864 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cbc63-3907-45e7-a2ed-0eb0b7d8541d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:08:20.388802 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-1c1cbc63-3907-45e7-a2ed-0eb0b7d8541d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:08:20.401022 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1cbc63-3907-45e7-a2ed-0eb0b7d8541d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:08:20.402586 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1cbc63-3907-45e7-a2ed-0eb0b7d8541d service nova] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:08:20.412959 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1173/2359] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:08:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:08:22.058410 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243a003-9d86-4a22-952f-1ef4fe50a5c3 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:08:22.058881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243a003-9d86-4a22-952f-1ef4fe50a5c3 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:08:22.058976 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243a003-9d86-4a22-952f-1ef4fe50a5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:08:22.079569 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-0243a003-9d86-4a22-952f-1ef4fe50a5c3 None None] MFA Rules not processed for user `1afe13c8fbb1468b8ef3d0d0c671208d`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:08:22.080873 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-0243a003-9d86-4a22-952f-1ef4fe50a5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:08:22.087846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1187/2360] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:08:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:08:22.472384 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1cbc63-3907-45e7-a2ed-0eb0b7d8541d service nova] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:08:22.484817 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1cbc63-3907-45e7-a2ed-0eb0b7d8541d service nova] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:08:22.489332 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d117c2bf-b88f-4c18-9d06-07304b4cc058 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:08:22.490216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117c2bf-b88f-4c18-9d06-07304b4cc058 service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:08:22.490707 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117c2bf-b88f-4c18-9d06-07304b4cc058 service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:08:22.491196 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117c2bf-b88f-4c18-9d06-07304b4cc058 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:08:22.493205 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-d117c2bf-b88f-4c18-9d06-07304b4cc058 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:08:22.504716 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d117c2bf-b88f-4c18-9d06-07304b4cc058 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:08:22.506390 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d117c2bf-b88f-4c18-9d06-07304b4cc058 service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:08:22.516797 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1174/2361] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:08:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:08:23.169436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0243a003-9d86-4a22-952f-1ef4fe50a5c3 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:08:23.171511 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-0243a003-9d86-4a22-952f-1ef4fe50a5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:08:23.256917 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0243a003-9d86-4a22-952f-1ef4fe50a5c3 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:08:23.262104 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217746a-7522-413b-b4e9-14294a35809a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61e2c954f9a49abad9f3de17ee39f63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:08:23.262985 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217746a-7522-413b-b4e9-14294a35809a service placement] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:08:23.263739 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217746a-7522-413b-b4e9-14294a35809a service placement] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:08:23.264610 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217746a-7522-413b-b4e9-14294a35809a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:08:23.266521 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-a217746a-7522-413b-b4e9-14294a35809a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:08:23.281382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217746a-7522-413b-b4e9-14294a35809a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:08:23.283639 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217746a-7522-413b-b4e9-14294a35809a service placement] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:08:23.297777 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1188/2362] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:08:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:08:29.534431 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73d686-f0f1-4bd2-ba2f-2572d85a23d9 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:08:29.534943 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73d686-f0f1-4bd2-ba2f-2572d85a23d9 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:08:29.535030 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73d686-f0f1-4bd2-ba2f-2572d85a23d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:08:29.554864 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-ac73d686-f0f1-4bd2-ba2f-2572d85a23d9 None None] MFA Rules not processed for user `3488a7d2258242c0859a105c1d162767`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:08:29.556103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-ac73d686-f0f1-4bd2-ba2f-2572d85a23d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:08:29.563758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1175/2363] 10.222.0.39 () {58 vars in 1009 bytes} [Fri Aug 5 20:08:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:08:29.605883 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217746a-7522-413b-b4e9-14294a35809a service placement] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:08:29.618016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217746a-7522-413b-b4e9-14294a35809a service placement] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:08:29.625531 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0780f-6177-4f79-8782-4e2924c71828 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91d3d43cb0bd4ca3b8e8f10e29cc421c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:08:29.626465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0780f-6177-4f79-8782-4e2924c71828 service swift] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:08:29.626957 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0780f-6177-4f79-8782-4e2924c71828 service swift] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:08:29.627283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0780f-6177-4f79-8782-4e2924c71828 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:08:29.638838 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0780f-6177-4f79-8782-4e2924c71828 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:08:29.641130 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0780f-6177-4f79-8782-4e2924c71828 service swift] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:08:29.652993 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1189/2364] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Aug 5 20:08:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 20:08:33.601061 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac73d686-f0f1-4bd2-ba2f-2572d85a23d9 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:08:33.622092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac73d686-f0f1-4bd2-ba2f-2572d85a23d9 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:08:33.634482 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b7029-9c9b-4840-b5d7-c9500e1d8287 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:08:33.636097 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b7029-9c9b-4840-b5d7-c9500e1d8287 service nova] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:08:33.636644 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b7029-9c9b-4840-b5d7-c9500e1d8287 service nova] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:08:33.637134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b7029-9c9b-4840-b5d7-c9500e1d8287 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:08:33.642879 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-ff1b7029-9c9b-4840-b5d7-c9500e1d8287 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:08:33.687079 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b7029-9c9b-4840-b5d7-c9500e1d8287 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:08:33.693389 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b7029-9c9b-4840-b5d7-c9500e1d8287 service nova] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:08:33.710405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1176/2365] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:08:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:08:39.646799 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=41 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:08:43.702020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=21 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:08:44.461569 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0780f-6177-4f79-8782-4e2924c71828 service swift] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:08:44.464227 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-41c0780f-6177-4f79-8782-4e2924c71828 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:08:44.530714 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0780f-6177-4f79-8782-4e2924c71828 service swift] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:08:44.538301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25035b2-162b-4064-803d-1e0b21a7a374 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:08:44.539311 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25035b2-162b-4064-803d-1e0b21a7a374 service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:08:44.539689 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25035b2-162b-4064-803d-1e0b21a7a374 service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:08:44.540135 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25035b2-162b-4064-803d-1e0b21a7a374 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:08:44.542077 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e25035b2-162b-4064-803d-1e0b21a7a374 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:08:44.614839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25035b2-162b-4064-803d-1e0b21a7a374 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:08:44.616835 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25035b2-162b-4064-803d-1e0b21a7a374 service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:08:44.627925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1190/2366] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:08:44 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:08:46.543324 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b7029-9c9b-4840-b5d7-c9500e1d8287 service nova] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:08:46.556410 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b7029-9c9b-4840-b5d7-c9500e1d8287 service nova] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:08:46.563546 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2ff60-ee9d-4b58-b9ec-98e0ac80d841 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:08:46.564173 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2ff60-ee9d-4b58-b9ec-98e0ac80d841 service nova] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:08:46.564441 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2ff60-ee9d-4b58-b9ec-98e0ac80d841 service nova] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:08:46.564745 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2ff60-ee9d-4b58-b9ec-98e0ac80d841 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:08:46.566524 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-fda2ff60-ee9d-4b58-b9ec-98e0ac80d841 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:08:46.582763 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda2ff60-ee9d-4b58-b9ec-98e0ac80d841 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:08:46.584225 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda2ff60-ee9d-4b58-b9ec-98e0ac80d841 service nova] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:08:46.596425 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1177/2367] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:08:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:08:48.025740 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4920a-c5e8-495c-8775-429c8acde2ef None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:08:48.026268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4920a-c5e8-495c-8775-429c8acde2ef None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:08:48.026268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4920a-c5e8-495c-8775-429c8acde2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:08:48.053013 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-86f4920a-c5e8-495c-8775-429c8acde2ef None None] MFA Rules not processed for user `3488a7d2258242c0859a105c1d162767`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:08:48.054201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-86f4920a-c5e8-495c-8775-429c8acde2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:08:48.061478 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1191/2368] 10.222.0.39 () {58 vars in 967 bytes} [Fri Aug 5 20:08:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:08:48.074180 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2ff60-ee9d-4b58-b9ec-98e0ac80d841 service nova] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:08:48.085497 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2ff60-ee9d-4b58-b9ec-98e0ac80d841 service nova] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:08:48.090466 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614756c1-6f3a-4720-a110-d13e705ac858 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91d3d43cb0bd4ca3b8e8f10e29cc421c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:08:48.091575 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614756c1-6f3a-4720-a110-d13e705ac858 service swift] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:08:48.091859 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614756c1-6f3a-4720-a110-d13e705ac858 service swift] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:08:48.092131 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614756c1-6f3a-4720-a110-d13e705ac858 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:08:48.105984 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614756c1-6f3a-4720-a110-d13e705ac858 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:08:48.108748 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614756c1-6f3a-4720-a110-d13e705ac858 service swift] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:08:48.123576 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1178/2369] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Aug 5 20:08:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 20:08:49.994222 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4920a-c5e8-495c-8775-429c8acde2ef None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:08:49.995642 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-86f4920a-c5e8-495c-8775-429c8acde2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:08:50.010937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4920a-c5e8-495c-8775-429c8acde2ef None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:08:50.016954 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5cda5-2d7f-4e19-90c4-7647194b6e01 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61e2c954f9a49abad9f3de17ee39f63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:08:50.017568 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5cda5-2d7f-4e19-90c4-7647194b6e01 service placement] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:08:50.017844 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5cda5-2d7f-4e19-90c4-7647194b6e01 service placement] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:08:50.018117 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5cda5-2d7f-4e19-90c4-7647194b6e01 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:08:50.019845 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5dc5cda5-2d7f-4e19-90c4-7647194b6e01 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:08:50.032990 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc5cda5-2d7f-4e19-90c4-7647194b6e01 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:08:50.034779 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc5cda5-2d7f-4e19-90c4-7647194b6e01 service placement] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:08:50.050045 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614756c1-6f3a-4720-a110-d13e705ac858 service swift] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:08:50.050564 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1192/2370] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:08:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:08:50.051870 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-614756c1-6f3a-4720-a110-d13e705ac858 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:08:50.119821 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614756c1-6f3a-4720-a110-d13e705ac858 service swift] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:08:50.125349 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f8154-2a33-4562-8b2e-39e230388468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:08:50.125949 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f8154-2a33-4562-8b2e-39e230388468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:08:50.126227 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f8154-2a33-4562-8b2e-39e230388468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:08:50.126502 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f8154-2a33-4562-8b2e-39e230388468 admin admin] PATH_INFO: `/v3/users/a0dbaac800624b85ab8c07a369cf0aff` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:08:50.127655 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f8154-2a33-4562-8b2e-39e230388468 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0dbaac800624b85ab8c07a369cf0aff)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:08:50.129615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f8154-2a33-4562-8b2e-39e230388468 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:08:50.179344 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-d59f8154-2a33-4562-8b2e-39e230388468 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0dbaac800624b85ab8c07a369cf0aff'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:08:50.187334 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-d59f8154-2a33-4562-8b2e-39e230388468 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0dbaac800624b85ab8c07a369cf0aff'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:08:50.199111 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-d59f8154-2a33-4562-8b2e-39e230388468 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0dbaac800624b85ab8c07a369cf0aff'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:08:50.199751 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-d59f8154-2a33-4562-8b2e-39e230388468 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0dbaac800624b85ab8c07a369cf0aff'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:08:50.207127 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-d59f8154-2a33-4562-8b2e-39e230388468 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0dbaac800624b85ab8c07a369cf0aff'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:08:50.212952 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1179/2371] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:08:50 2022] DELETE /identity/v3/users/a0dbaac800624b85ab8c07a369cf0aff => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:08:50.401107 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5cda5-2d7f-4e19-90c4-7647194b6e01 service placement] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:08:50.411842 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5cda5-2d7f-4e19-90c4-7647194b6e01 service placement] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:08:50.470590 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94fe194-5993-4ad2-924d-ad769ff5bf01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:08:50.471323 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fe194-5993-4ad2-924d-ad769ff5bf01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:08:50.471525 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fe194-5993-4ad2-924d-ad769ff5bf01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:08:50.471762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fe194-5993-4ad2-924d-ad769ff5bf01 admin admin] PATH_INFO: `/v3/projects/2fee7422708346709cc0524d685031b3` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:08:50.473037 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94fe194-5993-4ad2-924d-ad769ff5bf01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fee7422708346709cc0524d685031b3)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:08:50.474919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94fe194-5993-4ad2-924d-ad769ff5bf01 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:08:50.514352 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:08:50.523780 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-b94fe194-5993-4ad2-924d-ad769ff5bf01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fee7422708346709cc0524d685031b3', 'request_id': 'req-b94fe194-5993-4ad2-924d-ad769ff5bf01'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:08:50.529935 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-b94fe194-5993-4ad2-924d-ad769ff5bf01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fee7422708346709cc0524d685031b3', 'request_id': 'req-b94fe194-5993-4ad2-924d-ad769ff5bf01'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:08:50.537054 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-b94fe194-5993-4ad2-924d-ad769ff5bf01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fee7422708346709cc0524d685031b3', 'request_id': 'req-b94fe194-5993-4ad2-924d-ad769ff5bf01'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:08:50.547888 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-b94fe194-5993-4ad2-924d-ad769ff5bf01 admin admin] The token cache is being invalidate because project 2fee7422708346709cc0524d685031b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:08:50.548172 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-b94fe194-5993-4ad2-924d-ad769ff5bf01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:08:50.548823 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-b94fe194-5993-4ad2-924d-ad769ff5bf01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:08:50.550424 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1193/2372] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:08:50 2022] DELETE /identity/v3/projects/2fee7422708346709cc0524d685031b3 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:08:59.039115 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f8154-2a33-4562-8b2e-39e230388468 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:08:59.040537 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-d59f8154-2a33-4562-8b2e-39e230388468 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:08:59.098637 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f8154-2a33-4562-8b2e-39e230388468 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:08:59.109867 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:08:59.163794 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f88b1e5-10e9-4df1-a959-5b47c0d5afb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:08:59.164409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88b1e5-10e9-4df1-a959-5b47c0d5afb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:08:59.164757 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88b1e5-10e9-4df1-a959-5b47c0d5afb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:08:59.165037 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88b1e5-10e9-4df1-a959-5b47c0d5afb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:08:59.166855 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-8f88b1e5-10e9-4df1-a959-5b47c0d5afb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:08:59.228539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f88b1e5-10e9-4df1-a959-5b47c0d5afb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:08:59.229943 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f88b1e5-10e9-4df1-a959-5b47c0d5afb0 service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:08:59.239560 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1180/2373] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:08:59 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:09:00.544875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=11,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:09:05.445734 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94fe194-5993-4ad2-924d-ad769ff5bf01 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:09:05.447230 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-b94fe194-5993-4ad2-924d-ad769ff5bf01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:09:05.459880 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94fe194-5993-4ad2-924d-ad769ff5bf01 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:09:05.463402 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3392fbde-c159-4594-86c8-c36b388dc6e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:09:05.464001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392fbde-c159-4594-86c8-c36b388dc6e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:05.464285 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392fbde-c159-4594-86c8-c36b388dc6e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:05.464555 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392fbde-c159-4594-86c8-c36b388dc6e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:05.466320 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-3392fbde-c159-4594-86c8-c36b388dc6e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:09:05.531758 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3392fbde-c159-4594-86c8-c36b388dc6e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:09:05.533238 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3392fbde-c159-4594-86c8-c36b388dc6e2 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:09:05.542806 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1194/2374] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:09:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:09:06.048972 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f88b1e5-10e9-4df1-a959-5b47c0d5afb0 service neutron] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:09:06.050461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-8f88b1e5-10e9-4df1-a959-5b47c0d5afb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:09:06.065594 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f88b1e5-10e9-4df1-a959-5b47c0d5afb0 service neutron] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:09:06.070088 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7098a7f5-8bea-4b02-902c-b0c2924a5c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:09:06.070720 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098a7f5-8bea-4b02-902c-b0c2924a5c11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:06.070999 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098a7f5-8bea-4b02-902c-b0c2924a5c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:06.071274 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098a7f5-8bea-4b02-902c-b0c2924a5c11 admin admin] PATH_INFO: `/v3/users/ac0a233c84e1413490057c64654abe99` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:06.072429 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7098a7f5-8bea-4b02-902c-b0c2924a5c11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac0a233c84e1413490057c64654abe99)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:09:06.075614 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7098a7f5-8bea-4b02-902c-b0c2924a5c11 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:09:06.137483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-7098a7f5-8bea-4b02-902c-b0c2924a5c11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac0a233c84e1413490057c64654abe99'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:09:06.152331 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-7098a7f5-8bea-4b02-902c-b0c2924a5c11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac0a233c84e1413490057c64654abe99'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:09:06.166135 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-7098a7f5-8bea-4b02-902c-b0c2924a5c11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac0a233c84e1413490057c64654abe99'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:09:06.166845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-7098a7f5-8bea-4b02-902c-b0c2924a5c11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac0a233c84e1413490057c64654abe99'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:09:06.174786 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-7098a7f5-8bea-4b02-902c-b0c2924a5c11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac0a233c84e1413490057c64654abe99'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:09:06.182122 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1181/2375] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:09:06 2022] DELETE /identity/v3/users/ac0a233c84e1413490057c64654abe99 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:09:06.362995 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3392fbde-c159-4594-86c8-c36b388dc6e2 service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:09:06.373383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3392fbde-c159-4594-86c8-c36b388dc6e2 service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:09:06.432782 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346c124-353b-4d46-ab1c-d33008a643fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:09:06.433422 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346c124-353b-4d46-ab1c-d33008a643fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:06.433784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346c124-353b-4d46-ab1c-d33008a643fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:06.434209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346c124-353b-4d46-ab1c-d33008a643fe admin admin] PATH_INFO: `/v3/projects/7a5caf50650e45fbb1872e277f56e3b3` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:06.435587 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d346c124-353b-4d46-ab1c-d33008a643fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a5caf50650e45fbb1872e277f56e3b3)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:09:06.437565 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d346c124-353b-4d46-ab1c-d33008a643fe admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:09:06.484049 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-d346c124-353b-4d46-ab1c-d33008a643fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a5caf50650e45fbb1872e277f56e3b3', 'request_id': 'req-d346c124-353b-4d46-ab1c-d33008a643fe'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:09:06.490372 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-d346c124-353b-4d46-ab1c-d33008a643fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a5caf50650e45fbb1872e277f56e3b3', 'request_id': 'req-d346c124-353b-4d46-ab1c-d33008a643fe'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:09:06.497333 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-d346c124-353b-4d46-ab1c-d33008a643fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a5caf50650e45fbb1872e277f56e3b3', 'request_id': 'req-d346c124-353b-4d46-ab1c-d33008a643fe'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:09:06.507708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-d346c124-353b-4d46-ab1c-d33008a643fe admin admin] The token cache is being invalidate because project 7a5caf50650e45fbb1872e277f56e3b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:09:06.507996 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-d346c124-353b-4d46-ab1c-d33008a643fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:09:06.508646 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-d346c124-353b-4d46-ab1c-d33008a643fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:09:06.510132 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1195/2376] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:09:06 2022] DELETE /identity/v3/projects/7a5caf50650e45fbb1872e277f56e3b3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:09:06.516147 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbb8c7-0169-4a30-896d-913130f408be None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:06.516684 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbb8c7-0169-4a30-896d-913130f408be None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:06.517034 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbb8c7-0169-4a30-896d-913130f408be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:06.537142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-51dbb8c7-0169-4a30-896d-913130f408be None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:09:06.538324 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-51dbb8c7-0169-4a30-896d-913130f408be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:09:06.563895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:09:06.599439 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1182/2377] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:09:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:09:06.608631 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580f8b7-a6c7-4435-b8b2-0f65b89b50b1 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:06.608942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580f8b7-a6c7-4435-b8b2-0f65b89b50b1 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:06.609204 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580f8b7-a6c7-4435-b8b2-0f65b89b50b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:06.629214 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-b580f8b7-a6c7-4435-b8b2-0f65b89b50b1 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:09:06.630417 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-b580f8b7-a6c7-4435-b8b2-0f65b89b50b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:09:06.639459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1196/2378] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:09:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:09:06.643745 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbb8c7-0169-4a30-896d-913130f408be None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:09:06.653616 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbb8c7-0169-4a30-896d-913130f408be None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:09:06.658178 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ebf77-e842-4149-b40a-34dce79c4254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:09:06.658771 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ebf77-e842-4149-b40a-34dce79c4254 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:06.659052 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ebf77-e842-4149-b40a-34dce79c4254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:06.659322 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ebf77-e842-4149-b40a-34dce79c4254 admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:06.660008 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27ebf77-e842-4149-b40a-34dce79c4254 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:09:06.660313 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27ebf77-e842-4149-b40a-34dce79c4254 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:09:06.661953 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27ebf77-e842-4149-b40a-34dce79c4254 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:09:06.673820 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1183/2379] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:09:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:09:06.678105 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b580f8b7-a6c7-4435-b8b2-0f65b89b50b1 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:09:06.687854 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b580f8b7-a6c7-4435-b8b2-0f65b89b50b1 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:09:06.692493 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222b17b-9079-4844-9e8a-2284838cd8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:09:06.693132 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222b17b-9079-4844-9e8a-2284838cd8ff admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:06.693416 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222b17b-9079-4844-9e8a-2284838cd8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:06.693692 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222b17b-9079-4844-9e8a-2284838cd8ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:06.694356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222b17b-9079-4844-9e8a-2284838cd8ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:09:06.696553 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222b17b-9079-4844-9e8a-2284838cd8ff admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:09:06.713050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:09:06.727823 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1197/2380] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:09:06 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:09:06.732037 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ebf77-e842-4149-b40a-34dce79c4254 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:09:06.742260 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ebf77-e842-4149-b40a-34dce79c4254 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:09:06.797859 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165dac1e-496d-4d7e-a5eb-9fb64b1b524b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:09:06.798464 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dac1e-496d-4d7e-a5eb-9fb64b1b524b admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:06.798744 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dac1e-496d-4d7e-a5eb-9fb64b1b524b admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:06.799019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dac1e-496d-4d7e-a5eb-9fb64b1b524b admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:06.799667 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165dac1e-496d-4d7e-a5eb-9fb64b1b524b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:09:06.801641 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165dac1e-496d-4d7e-a5eb-9fb64b1b524b admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:09:06.826590 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1184/2381] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:09:06 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:09:06.827833 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222b17b-9079-4844-9e8a-2284838cd8ff admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:09:06.837904 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222b17b-9079-4844-9e8a-2284838cd8ff admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:09:06.842979 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa6c4ef-ad3b-46ca-84cc-c61dd20dd735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:09:06.843575 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6c4ef-ad3b-46ca-84cc-c61dd20dd735 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:06.843876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6c4ef-ad3b-46ca-84cc-c61dd20dd735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:06.844150 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6c4ef-ad3b-46ca-84cc-c61dd20dd735 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:06.844861 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa6c4ef-ad3b-46ca-84cc-c61dd20dd735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:09:06.846430 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa6c4ef-ad3b-46ca-84cc-c61dd20dd735 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:09:06.856441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1198/2382] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:09:06.860316 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165dac1e-496d-4d7e-a5eb-9fb64b1b524b admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:09:06.870147 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165dac1e-496d-4d7e-a5eb-9fb64b1b524b admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:09:06.875523 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e44c49-87ae-480b-bfb7-d7227ae9305a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:09:06.876123 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e44c49-87ae-480b-bfb7-d7227ae9305a admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:06.876403 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e44c49-87ae-480b-bfb7-d7227ae9305a admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:06.876732 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e44c49-87ae-480b-bfb7-d7227ae9305a admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:06.877424 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e44c49-87ae-480b-bfb7-d7227ae9305a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:09:06.878992 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e44c49-87ae-480b-bfb7-d7227ae9305a admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:09:06.890453 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1185/2383] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:09:06.894810 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa6c4ef-ad3b-46ca-84cc-c61dd20dd735 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:09:06.905042 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa6c4ef-ad3b-46ca-84cc-c61dd20dd735 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:09:06.909589 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b109d-ba98-407f-98b9-2bacf88525ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:09:06.910242 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b109d-ba98-407f-98b9-2bacf88525ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:06.910525 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b109d-ba98-407f-98b9-2bacf88525ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:06.910799 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b109d-ba98-407f-98b9-2bacf88525ce admin admin] PATH_INFO: `/v3/projects/cd1707d3a6c640b78baeb522cfb7ecdb/users/b5d959f3af3745ed8e0bc4a67faf34f1/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:06.925767 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93b109d-ba98-407f-98b9-2bacf88525ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd1707d3a6c640b78baeb522cfb7ecdb, user_id=b5d959f3af3745ed8e0bc4a67faf34f1, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:09:06.928395 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93b109d-ba98-407f-98b9-2bacf88525ce admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:09:06.941742 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1199/2384] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:09:06 2022] PUT /identity/v3/projects/cd1707d3a6c640b78baeb522cfb7ecdb/users/b5d959f3af3745ed8e0bc4a67faf34f1/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:09:06.946754 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5653a5a-d38e-431f-b86a-5c72fe9623e1 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:06.947043 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5653a5a-d38e-431f-b86a-5c72fe9623e1 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:06.947322 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5653a5a-d38e-431f-b86a-5c72fe9623e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:06.966479 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-c5653a5a-d38e-431f-b86a-5c72fe9623e1 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:09:06.967689 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-c5653a5a-d38e-431f-b86a-5c72fe9623e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:09:07.025795 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1186/2385] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:09:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:09:07.035351 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b109d-ba98-407f-98b9-2bacf88525ce admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:09:07.037064 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-a93b109d-ba98-407f-98b9-2bacf88525ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:09:07.095661 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b109d-ba98-407f-98b9-2bacf88525ce admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:09:07.099276 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de771f1-0181-40d0-85e4-29976fd626de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:09:07.099881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de771f1-0181-40d0-85e4-29976fd626de service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:07.100159 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de771f1-0181-40d0-85e4-29976fd626de service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:07.100430 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de771f1-0181-40d0-85e4-29976fd626de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:07.110141 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de771f1-0181-40d0-85e4-29976fd626de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:09:07.111557 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de771f1-0181-40d0-85e4-29976fd626de service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:09:07.121851 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1200/2386] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:09:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:09:11.425296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b1450-74ec-4f2f-a6be-84f509241401 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:11.425865 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b1450-74ec-4f2f-a6be-84f509241401 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:11.425865 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b1450-74ec-4f2f-a6be-84f509241401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:11.468355 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-a01b1450-74ec-4f2f-a6be-84f509241401 None None] MFA Rules not processed for user `b5d959f3af3745ed8e0bc4a67faf34f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:09:11.469609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-a01b1450-74ec-4f2f-a6be-84f509241401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:09:11.502487 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:09:11.553661 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1187/2387] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:09:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:09:11.558387 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de771f1-0181-40d0-85e4-29976fd626de service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:09:11.568800 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de771f1-0181-40d0-85e4-29976fd626de service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:09:11.573477 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f906c7-4d62-46e7-bd43-be137b6c160d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:09:11.574599 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f906c7-4d62-46e7-bd43-be137b6c160d admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:11.574973 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f906c7-4d62-46e7-bd43-be137b6c160d admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:11.575255 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f906c7-4d62-46e7-bd43-be137b6c160d admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:11.576076 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f906c7-4d62-46e7-bd43-be137b6c160d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:09:11.578255 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f906c7-4d62-46e7-bd43-be137b6c160d admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:09:11.611054 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1201/2388] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:09:11 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:09:11.615259 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b1450-74ec-4f2f-a6be-84f509241401 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:09:11.626552 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b1450-74ec-4f2f-a6be-84f509241401 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:09:11.686027 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad69ee0-5b44-4dc5-97a3-423d800400d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:09:11.686845 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad69ee0-5b44-4dc5-97a3-423d800400d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:11.687131 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad69ee0-5b44-4dc5-97a3-423d800400d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:11.687406 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad69ee0-5b44-4dc5-97a3-423d800400d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:11.688118 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad69ee0-5b44-4dc5-97a3-423d800400d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:09:11.690092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad69ee0-5b44-4dc5-97a3-423d800400d2 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:09:11.716990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1188/2389] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:09:11 2022] POST /identity/v3/users => generated 296 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:09:11.721213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f906c7-4d62-46e7-bd43-be137b6c160d admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:09:11.733884 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f906c7-4d62-46e7-bd43-be137b6c160d admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:09:11.738880 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877720d7-5b62-478e-9c35-e0d23e1633a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:09:11.739510 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877720d7-5b62-478e-9c35-e0d23e1633a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:11.739791 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877720d7-5b62-478e-9c35-e0d23e1633a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:11.740062 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877720d7-5b62-478e-9c35-e0d23e1633a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:11.740832 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877720d7-5b62-478e-9c35-e0d23e1633a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:09:11.742441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877720d7-5b62-478e-9c35-e0d23e1633a1 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:09:11.754163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1202/2390] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:09:11.758471 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad69ee0-5b44-4dc5-97a3-423d800400d2 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:09:11.770185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad69ee0-5b44-4dc5-97a3-423d800400d2 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:09:11.774841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d43ff8-cc4f-4439-87e5-74a14415eb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:09:11.775535 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d43ff8-cc4f-4439-87e5-74a14415eb66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:11.775902 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d43ff8-cc4f-4439-87e5-74a14415eb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:11.776186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d43ff8-cc4f-4439-87e5-74a14415eb66 admin admin] PATH_INFO: `/v3/projects/b70d4c679d90460996b6e0bdfa4ea50a/users/901fcc9fa690437f9cc2a30f6f4204ac/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:11.793215 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d43ff8-cc4f-4439-87e5-74a14415eb66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b70d4c679d90460996b6e0bdfa4ea50a, user_id=901fcc9fa690437f9cc2a30f6f4204ac, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:09:11.796118 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d43ff8-cc4f-4439-87e5-74a14415eb66 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:09:11.829090 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1189/2391] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:09:11 2022] PUT /identity/v3/projects/b70d4c679d90460996b6e0bdfa4ea50a/users/901fcc9fa690437f9cc2a30f6f4204ac/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:09:11.833187 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877720d7-5b62-478e-9c35-e0d23e1633a1 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:09:11.844156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877720d7-5b62-478e-9c35-e0d23e1633a1 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:09:11.909041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b588719-1e51-43ed-974d-6accd0a5173d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:09:11.909671 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b588719-1e51-43ed-974d-6accd0a5173d admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:11.909955 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b588719-1e51-43ed-974d-6accd0a5173d admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:11.910235 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b588719-1e51-43ed-974d-6accd0a5173d admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:11.910940 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b588719-1e51-43ed-974d-6accd0a5173d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:09:11.912545 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b588719-1e51-43ed-974d-6accd0a5173d admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:09:11.924163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1203/2392] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:09:11.928361 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d43ff8-cc4f-4439-87e5-74a14415eb66 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:09:11.940491 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d43ff8-cc4f-4439-87e5-74a14415eb66 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:09:11.945360 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa6b93-b4b2-46df-a803-ddfa2347cc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:09:11.946042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa6b93-b4b2-46df-a803-ddfa2347cc66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:11.946320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa6b93-b4b2-46df-a803-ddfa2347cc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:11.946597 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa6b93-b4b2-46df-a803-ddfa2347cc66 admin admin] PATH_INFO: `/v3/projects/b70d4c679d90460996b6e0bdfa4ea50a/users/901fcc9fa690437f9cc2a30f6f4204ac/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:11.948603 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa6b93-b4b2-46df-a803-ddfa2347cc66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b70d4c679d90460996b6e0bdfa4ea50a, user_id=901fcc9fa690437f9cc2a30f6f4204ac, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:09:11.951453 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa6b93-b4b2-46df-a803-ddfa2347cc66 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:09:11.962653 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1190/2393] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:09:11 2022] PUT /identity/v3/projects/b70d4c679d90460996b6e0bdfa4ea50a/users/901fcc9fa690437f9cc2a30f6f4204ac/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:09:16.488134 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac6f4a0-d30a-42d5-a04e-95403a14d402 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:16.488624 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac6f4a0-d30a-42d5-a04e-95403a14d402 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:16.488723 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac6f4a0-d30a-42d5-a04e-95403a14d402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:16.518009 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-4ac6f4a0-d30a-42d5-a04e-95403a14d402 None None] MFA Rules not processed for user `901fcc9fa690437f9cc2a30f6f4204ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:09:16.519234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-4ac6f4a0-d30a-42d5-a04e-95403a14d402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:09:16.533291 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:09:16.595751 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1204/2394] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:09:16 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:09:16.609772 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa6b93-b4b2-46df-a803-ddfa2347cc66 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:09:16.611465 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-dcfa6b93-b4b2-46df-a803-ddfa2347cc66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:09:16.705865 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa6b93-b4b2-46df-a803-ddfa2347cc66 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:09:16.713652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c45c7-6be2-4c6f-8d9d-170fe026486d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:09:16.714888 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c45c7-6be2-4c6f-8d9d-170fe026486d service nova] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:16.715223 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c45c7-6be2-4c6f-8d9d-170fe026486d service nova] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:16.715511 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c45c7-6be2-4c6f-8d9d-170fe026486d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:16.730559 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:09:16.776765 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c45c7-6be2-4c6f-8d9d-170fe026486d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:09:16.779066 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c45c7-6be2-4c6f-8d9d-170fe026486d service nova] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:09:16.788163 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1191/2395] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:09:16 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:09:24.080327 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452e5f5-6821-4ef5-920b-7527fb123cda None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:24.080800 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452e5f5-6821-4ef5-920b-7527fb123cda None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:24.080942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452e5f5-6821-4ef5-920b-7527fb123cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:24.100733 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-a452e5f5-6821-4ef5-920b-7527fb123cda None None] MFA Rules not processed for user `3488a7d2258242c0859a105c1d162767`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:09:24.101946 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-a452e5f5-6821-4ef5-920b-7527fb123cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:09:24.161963 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1205/2396] 10.222.0.39 () {58 vars in 1009 bytes} [Fri Aug 5 20:09:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:09:24.174508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c45c7-6be2-4c6f-8d9d-170fe026486d service nova] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:09:24.175889 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-a22c45c7-6be2-4c6f-8d9d-170fe026486d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:09:24.246186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c45c7-6be2-4c6f-8d9d-170fe026486d service nova] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:09:24.253320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176da05f-5d0a-49e9-8939-5798a170b383 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91d3d43cb0bd4ca3b8e8f10e29cc421c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:09:24.254312 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176da05f-5d0a-49e9-8939-5798a170b383 service swift] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:24.254591 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176da05f-5d0a-49e9-8939-5798a170b383 service swift] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:24.254871 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176da05f-5d0a-49e9-8939-5798a170b383 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:24.265398 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176da05f-5d0a-49e9-8939-5798a170b383 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:09:24.266973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176da05f-5d0a-49e9-8939-5798a170b383 service swift] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:09:24.275037 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1192/2397] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Aug 5 20:09:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 20:09:25.037535 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452e5f5-6821-4ef5-920b-7527fb123cda None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:09:25.039193 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-a452e5f5-6821-4ef5-920b-7527fb123cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:09:25.056222 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452e5f5-6821-4ef5-920b-7527fb123cda None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:09:25.063647 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34d047-4820-441d-a39e-095601f6e36d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:09:25.064359 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34d047-4820-441d-a39e-095601f6e36d service nova] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:25.064766 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34d047-4820-441d-a39e-095601f6e36d service nova] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:25.065162 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34d047-4820-441d-a39e-095601f6e36d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:25.067157 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-ee34d047-4820-441d-a39e-095601f6e36d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:09:25.156797 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34d047-4820-441d-a39e-095601f6e36d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:09:25.158481 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34d047-4820-441d-a39e-095601f6e36d service nova] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:09:25.172146 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1206/2398] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:09:25 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:09:34.271784 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=52 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:09:35.165244 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=75 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:09:41.577649 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176da05f-5d0a-49e9-8939-5798a170b383 service swift] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:09:41.631268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176da05f-5d0a-49e9-8939-5798a170b383 service swift] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:09:41.635319 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea3b14-121e-4bd4-ab10-008b1a8b352a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:09:41.636326 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea3b14-121e-4bd4-ab10-008b1a8b352a service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:41.636631 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea3b14-121e-4bd4-ab10-008b1a8b352a service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:41.636908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea3b14-121e-4bd4-ab10-008b1a8b352a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:41.638706 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-99ea3b14-121e-4bd4-ab10-008b1a8b352a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:09:41.648041 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea3b14-121e-4bd4-ab10-008b1a8b352a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:09:41.649476 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea3b14-121e-4bd4-ab10-008b1a8b352a service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:09:41.658117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1193/2399] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:09:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:09:41.780878 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73b64ef-679d-49aa-833d-7a51af1a4798 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:41.781452 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73b64ef-679d-49aa-833d-7a51af1a4798 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:41.781576 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73b64ef-679d-49aa-833d-7a51af1a4798 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:41.782879 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1207/2400] 10.222.0.39 () {54 vars in 921 bytes} [Fri Aug 5 20:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 20:09:41.788562 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441ecda-1ac7-43a6-9993-6c195e064e18 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:41.788867 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441ecda-1ac7-43a6-9993-6c195e064e18 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:41.789142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441ecda-1ac7-43a6-9993-6c195e064e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:41.809304 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-7441ecda-1ac7-43a6-9993-6c195e064e18 None None] MFA Rules not processed for user `95e9f085e4f04d118373b8abff27543a`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:09:41.810507 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-7441ecda-1ac7-43a6-9993-6c195e064e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:09:41.819720 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1194/2401] 10.222.0.39 () {58 vars in 1018 bytes} [Fri Aug 5 20:09:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:09:41.832435 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73b64ef-679d-49aa-833d-7a51af1a4798 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:09:41.834264 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e73b64ef-679d-49aa-833d-7a51af1a4798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:09:41.896783 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73b64ef-679d-49aa-833d-7a51af1a4798 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:09:41.900503 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e94e3f-ae80-4e7f-9d7c-2ac3bb4cc58a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe69b398c1f4e18b625a63ae40b01e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:09:41.901328 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e94e3f-ae80-4e7f-9d7c-2ac3bb4cc58a service glance] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:41.901606 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e94e3f-ae80-4e7f-9d7c-2ac3bb4cc58a service glance] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:41.901905 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e94e3f-ae80-4e7f-9d7c-2ac3bb4cc58a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:41.912293 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e94e3f-ae80-4e7f-9d7c-2ac3bb4cc58a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:09:41.913750 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e94e3f-ae80-4e7f-9d7c-2ac3bb4cc58a service glance] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:09:41.924441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1208/2402] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:09:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:09:41.993208 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f8a72-91e9-4b84-abef-1894211acea5 None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:41.993491 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f8a72-91e9-4b84-abef-1894211acea5 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:41.993768 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f8a72-91e9-4b84-abef-1894211acea5 None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:41.995297 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1195/2403] 10.222.0.39 () {54 vars in 921 bytes} [Fri Aug 5 20:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 20:09:42.000001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d52375-1e94-40b8-8bff-0a164a678920 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:42.000283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d52375-1e94-40b8-8bff-0a164a678920 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:42.000568 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d52375-1e94-40b8-8bff-0a164a678920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:42.021524 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-25d52375-1e94-40b8-8bff-0a164a678920 None None] MFA Rules not processed for user `b1c75368c00e4b0cb7b3163728843479`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:09:42.022628 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-25d52375-1e94-40b8-8bff-0a164a678920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:09:42.031070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1209/2404] 10.222.0.39 () {58 vars in 1018 bytes} [Fri Aug 5 20:09:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:09:42.043526 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f8a72-91e9-4b84-abef-1894211acea5 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:09:42.054567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f8a72-91e9-4b84-abef-1894211acea5 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:09:42.058357 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513e686-cc65-476f-9a50-08a6d0efcba5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:09:42.058961 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513e686-cc65-476f-9a50-08a6d0efcba5 service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:42.059242 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513e686-cc65-476f-9a50-08a6d0efcba5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:42.059516 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513e686-cc65-476f-9a50-08a6d0efcba5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:42.066476 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6513e686-cc65-476f-9a50-08a6d0efcba5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:09:42.067884 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6513e686-cc65-476f-9a50-08a6d0efcba5 service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:09:42.075640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1196/2405] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:09:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:09:42.163090 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3bef56-f557-4ea7-9346-44ed41e33c12 None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:42.163373 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3bef56-f557-4ea7-9346-44ed41e33c12 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:42.163646 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3bef56-f557-4ea7-9346-44ed41e33c12 None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:42.165283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1210/2406] 10.222.0.39 () {54 vars in 921 bytes} [Fri Aug 5 20:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 20:09:42.173052 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235fe74-17ee-45f9-8d40-2509193d2dfa None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:42.173344 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235fe74-17ee-45f9-8d40-2509193d2dfa None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:42.173612 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235fe74-17ee-45f9-8d40-2509193d2dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:42.195632 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-b235fe74-17ee-45f9-8d40-2509193d2dfa None None] MFA Rules not processed for user `95e9f085e4f04d118373b8abff27543a`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:09:42.196890 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-b235fe74-17ee-45f9-8d40-2509193d2dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:09:42.207020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1197/2407] 10.222.0.39 () {58 vars in 1018 bytes} [Fri Aug 5 20:09:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:09:42.217623 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3bef56-f557-4ea7-9346-44ed41e33c12 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:09:42.219418 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-4e3bef56-f557-4ea7-9346-44ed41e33c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:09:42.233964 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3bef56-f557-4ea7-9346-44ed41e33c12 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:09:42.237834 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5989ad1-c199-4b76-acee-d1b5abe3820d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:09:42.238518 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5989ad1-c199-4b76-acee-d1b5abe3820d service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:42.238896 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5989ad1-c199-4b76-acee-d1b5abe3820d service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:42.239265 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5989ad1-c199-4b76-acee-d1b5abe3820d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:42.251157 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5989ad1-c199-4b76-acee-d1b5abe3820d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:09:42.252759 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5989ad1-c199-4b76-acee-d1b5abe3820d service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:09:42.264685 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1211/2408] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:09:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:09:44.982606 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12135222-71c4-4a4c-97d9-f64fda1d145b None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:44.983126 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12135222-71c4-4a4c-97d9-f64fda1d145b None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:44.983280 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12135222-71c4-4a4c-97d9-f64fda1d145b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:45.001974 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-12135222-71c4-4a4c-97d9-f64fda1d145b None None] MFA Rules not processed for user `3488a7d2258242c0859a105c1d162767`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:09:45.003156 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-12135222-71c4-4a4c-97d9-f64fda1d145b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:09:45.010201 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1198/2409] 10.222.0.39 () {58 vars in 1009 bytes} [Fri Aug 5 20:09:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:09:45.022482 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5989ad1-c199-4b76-acee-d1b5abe3820d service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:09:45.034242 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5989ad1-c199-4b76-acee-d1b5abe3820d service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:09:45.039270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e948b951-54f1-4372-b071-dc50d2a4d938 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91d3d43cb0bd4ca3b8e8f10e29cc421c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:09:45.039898 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948b951-54f1-4372-b071-dc50d2a4d938 service swift] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:45.040183 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948b951-54f1-4372-b071-dc50d2a4d938 service swift] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:45.040456 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948b951-54f1-4372-b071-dc50d2a4d938 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:45.049353 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e948b951-54f1-4372-b071-dc50d2a4d938 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:09:45.050931 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e948b951-54f1-4372-b071-dc50d2a4d938 service swift] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:09:45.059597 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1212/2410] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Aug 5 20:09:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 20:09:45.465866 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932696f1-71dc-49c3-8e47-0ea468b242d1 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:45.466152 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932696f1-71dc-49c3-8e47-0ea468b242d1 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:45.466425 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932696f1-71dc-49c3-8e47-0ea468b242d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:45.486149 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-932696f1-71dc-49c3-8e47-0ea468b242d1 None None] MFA Rules not processed for user `b5d959f3af3745ed8e0bc4a67faf34f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:09:45.487363 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-932696f1-71dc-49c3-8e47-0ea468b242d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:09:45.494609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1199/2411] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:09:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:09:53.837566 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e948b951-54f1-4372-b071-dc50d2a4d938 service swift] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:09:53.851581 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e948b951-54f1-4372-b071-dc50d2a4d938 service swift] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:09:53.855941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde3db9c-51f8-48d2-b84a-226b49911923 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:09:53.856595 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3db9c-51f8-48d2-b84a-226b49911923 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:09:53.856884 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3db9c-51f8-48d2-b84a-226b49911923 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:09:53.857157 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3db9c-51f8-48d2-b84a-226b49911923 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:09:53.859085 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-cde3db9c-51f8-48d2-b84a-226b49911923 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:09:53.923174 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde3db9c-51f8-48d2-b84a-226b49911923 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:09:53.925019 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde3db9c-51f8-48d2-b84a-226b49911923 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:09:53.935268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1213/2412] 10.222.0.39 () {60 vars in 1396 bytes} [Fri Aug 5 20:09:53 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3266 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:09:55.480513 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=44 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:10:03.929953 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=69 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:10:08.600182 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb25056-3dc0-4fec-aa71-40cc5e36cb8c None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:10:08.600789 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb25056-3dc0-4fec-aa71-40cc5e36cb8c None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:10:08.600789 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb25056-3dc0-4fec-aa71-40cc5e36cb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:10:08.621055 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-6cb25056-3dc0-4fec-aa71-40cc5e36cb8c None None] MFA Rules not processed for user `293b8a62f19c45c1bf5b897c7cf40edb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:10:08.622241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-6cb25056-3dc0-4fec-aa71-40cc5e36cb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:10:08.629165 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1200/2413] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:10:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:10:10.810601 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390b6ec-2780-4c50-bebf-97d6a3ca0471 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:10:10.811149 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390b6ec-2780-4c50-bebf-97d6a3ca0471 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:10:10.811149 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390b6ec-2780-4c50-bebf-97d6a3ca0471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:10:10.830590 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-1390b6ec-2780-4c50-bebf-97d6a3ca0471 None None] MFA Rules not processed for user `3488a7d2258242c0859a105c1d162767`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:10:10.831825 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-1390b6ec-2780-4c50-bebf-97d6a3ca0471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:10:10.839391 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1214/2414] 10.222.0.39 () {58 vars in 967 bytes} [Fri Aug 5 20:10:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:10:10.855487 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb25056-3dc0-4fec-aa71-40cc5e36cb8c None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:10:10.870099 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb25056-3dc0-4fec-aa71-40cc5e36cb8c None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:10:10.875302 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1ea1f-ee67-4972-bc4f-fa2183281bc2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91d3d43cb0bd4ca3b8e8f10e29cc421c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:10:10.875915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1ea1f-ee67-4972-bc4f-fa2183281bc2 service swift] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:10:10.876194 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1ea1f-ee67-4972-bc4f-fa2183281bc2 service swift] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:10:10.876466 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1ea1f-ee67-4972-bc4f-fa2183281bc2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:10:10.886855 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc1ea1f-ee67-4972-bc4f-fa2183281bc2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:10:10.888453 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc1ea1f-ee67-4972-bc4f-fa2183281bc2 service swift] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:10:10.897364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1201/2415] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Aug 5 20:10:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 20:10:20.825037 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:10:20.894117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=9 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:10:23.125616 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1390b6ec-2780-4c50-bebf-97d6a3ca0471 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:10:23.135820 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1390b6ec-2780-4c50-bebf-97d6a3ca0471 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:10:23.139369 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230420d-ec1d-4cf0-8c32-815cbaffe4ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:10:23.139968 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230420d-ec1d-4cf0-8c32-815cbaffe4ac service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:10:23.140244 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230420d-ec1d-4cf0-8c32-815cbaffe4ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:10:23.140509 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230420d-ec1d-4cf0-8c32-815cbaffe4ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:10:23.150077 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230420d-ec1d-4cf0-8c32-815cbaffe4ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:10:23.151498 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230420d-ec1d-4cf0-8c32-815cbaffe4ac service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:10:23.160832 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1215/2416] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:10:23 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:10:33.156722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=6 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:10:52.307877 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1ea1f-ee67-4972-bc4f-fa2183281bc2 service swift] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:10:52.393183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1ea1f-ee67-4972-bc4f-fa2183281bc2 service swift] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:10:52.400437 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd23a25-7240-4a8a-8c7c-0bb228b8c155 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:10:52.401030 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd23a25-7240-4a8a-8c7c-0bb228b8c155 service nova] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:10:52.401371 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd23a25-7240-4a8a-8c7c-0bb228b8c155 service nova] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:10:52.401629 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd23a25-7240-4a8a-8c7c-0bb228b8c155 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:10:52.403525 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-8bd23a25-7240-4a8a-8c7c-0bb228b8c155 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:10:52.502096 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd23a25-7240-4a8a-8c7c-0bb228b8c155 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:10:52.504657 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd23a25-7240-4a8a-8c7c-0bb228b8c155 service nova] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:10:52.518508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1202/2417] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:10:52 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:10:53.124084 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230420d-ec1d-4cf0-8c32-815cbaffe4ac service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:10:53.126164 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5230420d-ec1d-4cf0-8c32-815cbaffe4ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:10:53.303070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230420d-ec1d-4cf0-8c32-815cbaffe4ac service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:10:53.311443 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e409d0d-6368-4f82-9a8f-7731ee4340f1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61e2c954f9a49abad9f3de17ee39f63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:10:53.312333 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e409d0d-6368-4f82-9a8f-7731ee4340f1 service placement] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:10:53.312640 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e409d0d-6368-4f82-9a8f-7731ee4340f1 service placement] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:10:53.312917 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e409d0d-6368-4f82-9a8f-7731ee4340f1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:10:53.315112 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-3e409d0d-6368-4f82-9a8f-7731ee4340f1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:10:53.329866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e409d0d-6368-4f82-9a8f-7731ee4340f1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:10:53.331421 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e409d0d-6368-4f82-9a8f-7731ee4340f1 service placement] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:10:53.343347 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1216/2418] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:10:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:10:56.808695 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d47144b-6348-496e-98c6-bb5af22e342b None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:10:56.809289 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d47144b-6348-496e-98c6-bb5af22e342b None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:10:56.809289 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d47144b-6348-496e-98c6-bb5af22e342b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:10:56.828777 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-9d47144b-6348-496e-98c6-bb5af22e342b None None] MFA Rules not processed for user `1afe13c8fbb1468b8ef3d0d0c671208d`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:10:56.829905 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-9d47144b-6348-496e-98c6-bb5af22e342b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:10:56.886142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1203/2419] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:10:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:11:03.338222 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=55 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:11:06.878986 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=81 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:11:14.827278 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e706fa9d-b427-4f3d-846a-b83053c43b4b None None] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:11:14.827874 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e706fa9d-b427-4f3d-846a-b83053c43b4b None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:11:14.828021 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e706fa9d-b427-4f3d-846a-b83053c43b4b None None] PATH_INFO: `/` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:11:14.829487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1217/2420] 10.222.0.39 () {54 vars in 921 bytes} [Fri Aug 5 20:11:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 20:11:14.835338 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3fb58-e63b-4482-a074-0ea373067b3f None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:11:14.835786 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3fb58-e63b-4482-a074-0ea373067b3f None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:11:14.835903 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3fb58-e63b-4482-a074-0ea373067b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:11:14.889988 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-28f3fb58-e63b-4482-a074-0ea373067b3f None None] MFA Rules not processed for user `b1c75368c00e4b0cb7b3163728843479`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:11:14.891183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-28f3fb58-e63b-4482-a074-0ea373067b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:11:14.908082 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1204/2421] 10.222.0.39 () {58 vars in 1018 bytes} [Fri Aug 5 20:11:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:11:14.920057 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e706fa9d-b427-4f3d-846a-b83053c43b4b None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:11:14.931453 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e706fa9d-b427-4f3d-846a-b83053c43b4b None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:11:14.934959 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b63a13-058e-4b2d-8c03-2b03248c6618 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:11:14.935557 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b63a13-058e-4b2d-8c03-2b03248c6618 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:11:14.935833 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b63a13-058e-4b2d-8c03-2b03248c6618 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:11:14.936106 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b63a13-058e-4b2d-8c03-2b03248c6618 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:11:14.943574 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b63a13-058e-4b2d-8c03-2b03248c6618 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:11:14.945043 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b63a13-058e-4b2d-8c03-2b03248c6618 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:11:14.954299 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1218/2422] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:11:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:11:15.036662 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a89541-1754-4f30-aba1-4a645b9efdac None None] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:11:15.036956 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a89541-1754-4f30-aba1-4a645b9efdac None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:11:15.037237 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a89541-1754-4f30-aba1-4a645b9efdac None None] PATH_INFO: `/` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:11:15.038609 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1205/2423] 10.222.0.39 () {54 vars in 921 bytes} [Fri Aug 5 20:11:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 20:11:15.044688 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d70fe-fb5b-4dd8-a3fa-b5218dc36ce1 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:11:15.045050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d70fe-fb5b-4dd8-a3fa-b5218dc36ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:11:15.045329 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d70fe-fb5b-4dd8-a3fa-b5218dc36ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:11:15.076808 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-6e7d70fe-fb5b-4dd8-a3fa-b5218dc36ce1 None None] MFA Rules not processed for user `95e9f085e4f04d118373b8abff27543a`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:11:15.078006 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-6e7d70fe-fb5b-4dd8-a3fa-b5218dc36ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:11:15.088102 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1219/2424] 10.222.0.39 () {58 vars in 1018 bytes} [Fri Aug 5 20:11:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:11:15.098554 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a89541-1754-4f30-aba1-4a645b9efdac None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:11:15.110168 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a89541-1754-4f30-aba1-4a645b9efdac None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:11:15.116171 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3003e83-a7a5-427f-90e7-7b28435c9f36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:11:15.116864 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3003e83-a7a5-427f-90e7-7b28435c9f36 service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:11:15.117209 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3003e83-a7a5-427f-90e7-7b28435c9f36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:11:15.117434 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3003e83-a7a5-427f-90e7-7b28435c9f36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:11:15.128502 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3003e83-a7a5-427f-90e7-7b28435c9f36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:11:15.130011 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3003e83-a7a5-427f-90e7-7b28435c9f36 service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:11:15.141626 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1206/2425] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:11:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:11:25.071938 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=14 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:11:25.136907 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=26 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:11:43.792422 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d89f3d-94b6-40de-9983-c61816bd9102 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:11:43.793073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d89f3d-94b6-40de-9983-c61816bd9102 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:11:43.793073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d89f3d-94b6-40de-9983-c61816bd9102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:11:43.814513 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-48d89f3d-94b6-40de-9983-c61816bd9102 None None] MFA Rules not processed for user `160c88a7ba0144d4b1bc953767b12bb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:11:43.815770 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-48d89f3d-94b6-40de-9983-c61816bd9102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:11:43.829452 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1220/2426] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:11:43 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:11:53.808242 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:11:55.983216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3003e83-a7a5-427f-90e7-7b28435c9f36 service neutron] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:11:55.992735 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3003e83-a7a5-427f-90e7-7b28435c9f36 service neutron] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:11:55.996924 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8135c-fb0f-4cab-b194-1ed4be70b6ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:11:55.997572 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8135c-fb0f-4cab-b194-1ed4be70b6ce service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:11:55.998056 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8135c-fb0f-4cab-b194-1ed4be70b6ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:11:55.998526 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8135c-fb0f-4cab-b194-1ed4be70b6ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:11:56.001136 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-7de8135c-fb0f-4cab-b194-1ed4be70b6ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:11:56.092899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de8135c-fb0f-4cab-b194-1ed4be70b6ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:11:56.094332 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de8135c-fb0f-4cab-b194-1ed4be70b6ce service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:11:56.114287 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1207/2427] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:11:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:12:00.210243 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d89f3d-94b6-40de-9983-c61816bd9102 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:12:00.214656 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-48d89f3d-94b6-40de-9983-c61816bd9102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:12:00.241185 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d89f3d-94b6-40de-9983-c61816bd9102 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:12:00.247076 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e25b4-07a9-42c1-82ff-5fe95866ae8b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61e2c954f9a49abad9f3de17ee39f63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:12:00.248948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e25b4-07a9-42c1-82ff-5fe95866ae8b service placement] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:12:00.249403 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e25b4-07a9-42c1-82ff-5fe95866ae8b service placement] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:12:00.249843 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e25b4-07a9-42c1-82ff-5fe95866ae8b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:12:00.251934 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e74e25b4-07a9-42c1-82ff-5fe95866ae8b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:12:00.266684 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e25b4-07a9-42c1-82ff-5fe95866ae8b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:12:00.269476 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e25b4-07a9-42c1-82ff-5fe95866ae8b service placement] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:12:00.285948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1221/2428] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:12:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:12:00.299228 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8135c-fb0f-4cab-b194-1ed4be70b6ce service neutron] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:12:00.307249 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-7de8135c-fb0f-4cab-b194-1ed4be70b6ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:12:00.327445 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8135c-fb0f-4cab-b194-1ed4be70b6ce service neutron] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:12:00.334211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204f716-d392-4e86-8889-fc1b9d0a531e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:12:00.335360 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204f716-d392-4e86-8889-fc1b9d0a531e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:12:00.335847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204f716-d392-4e86-8889-fc1b9d0a531e admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:12:00.336317 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204f716-d392-4e86-8889-fc1b9d0a531e admin admin] PATH_INFO: `/v3/users/293b8a62f19c45c1bf5b897c7cf40edb` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:12:00.338233 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5204f716-d392-4e86-8889-fc1b9d0a531e admin admin] RBAC: Authorizing `identity:delete_user(user_id=293b8a62f19c45c1bf5b897c7cf40edb)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:12:00.341128 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5204f716-d392-4e86-8889-fc1b9d0a531e admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:12:00.457170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-5204f716-d392-4e86-8889-fc1b9d0a531e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '293b8a62f19c45c1bf5b897c7cf40edb'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:12:00.470445 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-5204f716-d392-4e86-8889-fc1b9d0a531e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '293b8a62f19c45c1bf5b897c7cf40edb'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:12:00.490530 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-5204f716-d392-4e86-8889-fc1b9d0a531e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '293b8a62f19c45c1bf5b897c7cf40edb'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:12:00.491541 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-5204f716-d392-4e86-8889-fc1b9d0a531e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '293b8a62f19c45c1bf5b897c7cf40edb'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:12:00.517749 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-5204f716-d392-4e86-8889-fc1b9d0a531e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '293b8a62f19c45c1bf5b897c7cf40edb'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:12:00.534090 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1208/2429] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:12:00 2022] DELETE /identity/v3/users/293b8a62f19c45c1bf5b897c7cf40edb => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:12:00.785332 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e25b4-07a9-42c1-82ff-5fe95866ae8b service placement] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:12:00.805355 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e25b4-07a9-42c1-82ff-5fe95866ae8b service placement] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:12:00.916859 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729e061-5582-4a08-aa50-1e9898c38201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:12:00.919535 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729e061-5582-4a08-aa50-1e9898c38201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:12:00.920043 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729e061-5582-4a08-aa50-1e9898c38201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:12:00.920334 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729e061-5582-4a08-aa50-1e9898c38201 admin admin] PATH_INFO: `/v3/projects/c5b5718607784ec9be3289f7916f70a6` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:12:00.921962 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2729e061-5582-4a08-aa50-1e9898c38201 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5b5718607784ec9be3289f7916f70a6)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:12:00.926803 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2729e061-5582-4a08-aa50-1e9898c38201 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:12:00.990165 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-2729e061-5582-4a08-aa50-1e9898c38201 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5b5718607784ec9be3289f7916f70a6', 'request_id': 'req-2729e061-5582-4a08-aa50-1e9898c38201'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:12:00.999493 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-2729e061-5582-4a08-aa50-1e9898c38201 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5b5718607784ec9be3289f7916f70a6', 'request_id': 'req-2729e061-5582-4a08-aa50-1e9898c38201'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:12:01.009299 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-2729e061-5582-4a08-aa50-1e9898c38201 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5b5718607784ec9be3289f7916f70a6', 'request_id': 'req-2729e061-5582-4a08-aa50-1e9898c38201'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:12:01.027139 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-2729e061-5582-4a08-aa50-1e9898c38201 admin admin] The token cache is being invalidate because project c5b5718607784ec9be3289f7916f70a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:12:01.027402 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-2729e061-5582-4a08-aa50-1e9898c38201 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:12:01.028249 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-2729e061-5582-4a08-aa50-1e9898c38201 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:12:01.036101 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1222/2430] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:12:00 2022] DELETE /identity/v3/projects/c5b5718607784ec9be3289f7916f70a6 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:12:01.037286 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0582535-94fc-4f83-885b-106d183fcfc7 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:12:01.037602 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0582535-94fc-4f83-885b-106d183fcfc7 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:12:01.041627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0582535-94fc-4f83-885b-106d183fcfc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:12:01.129815 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-f0582535-94fc-4f83-885b-106d183fcfc7 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:12:01.131307 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-f0582535-94fc-4f83-885b-106d183fcfc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:12:01.146791 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:12:01.223374 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1209/2431] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:12:01.234187 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc639d1-e186-4f73-8bc4-7015cc41e8fd None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:12:01.235332 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc639d1-e186-4f73-8bc4-7015cc41e8fd None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:12:01.236483 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc639d1-e186-4f73-8bc4-7015cc41e8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:12:01.260697 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-edc639d1-e186-4f73-8bc4-7015cc41e8fd None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:12:01.262594 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-edc639d1-e186-4f73-8bc4-7015cc41e8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:12:01.274143 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1223/2432] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:12:01.281462 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0582535-94fc-4f83-885b-106d183fcfc7 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:12:01.292038 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0582535-94fc-4f83-885b-106d183fcfc7 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:12:01.298338 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0101fe3-bdce-4af7-b6d1-8a90e40c7bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:12:01.299344 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0101fe3-bdce-4af7-b6d1-8a90e40c7bfe admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:12:01.299946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0101fe3-bdce-4af7-b6d1-8a90e40c7bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:12:01.300077 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0101fe3-bdce-4af7-b6d1-8a90e40c7bfe admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:12:01.301533 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0101fe3-bdce-4af7-b6d1-8a90e40c7bfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:12:01.301858 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0101fe3-bdce-4af7-b6d1-8a90e40c7bfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:12:01.303740 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0101fe3-bdce-4af7-b6d1-8a90e40c7bfe admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:12:01.323116 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc639d1-e186-4f73-8bc4-7015cc41e8fd None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:12:01.328180 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1210/2433] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:12:01.339914 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc639d1-e186-4f73-8bc4-7015cc41e8fd None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:12:01.346495 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4961a-4f89-492e-b671-af3efd97c21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:12:01.347889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4961a-4f89-492e-b671-af3efd97c21b admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:12:01.348607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4961a-4f89-492e-b671-af3efd97c21b admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:12:01.349292 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4961a-4f89-492e-b671-af3efd97c21b admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:12:01.350673 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4961a-4f89-492e-b671-af3efd97c21b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:12:01.354324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4961a-4f89-492e-b671-af3efd97c21b admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:12:01.398482 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1224/2434] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:12:01 2022] POST /identity/v3/projects => generated 377 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:12:01.406555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0101fe3-bdce-4af7-b6d1-8a90e40c7bfe admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:12:01.427860 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0101fe3-bdce-4af7-b6d1-8a90e40c7bfe admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:12:01.514061 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25179b3-052b-446c-a1cc-596242c2cf37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:12:01.515028 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25179b3-052b-446c-a1cc-596242c2cf37 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:12:01.515497 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25179b3-052b-446c-a1cc-596242c2cf37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:12:01.515989 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25179b3-052b-446c-a1cc-596242c2cf37 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:12:01.517128 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25179b3-052b-446c-a1cc-596242c2cf37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:12:01.520493 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25179b3-052b-446c-a1cc-596242c2cf37 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:12:01.562818 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1211/2435] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:12:01 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:12:01.564102 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4961a-4f89-492e-b671-af3efd97c21b admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:12:01.578755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4961a-4f89-492e-b671-af3efd97c21b admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:12:01.585041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecdf1e9-485f-4ec6-8cd8-bdcc9c6b99f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:12:01.585782 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecdf1e9-485f-4ec6-8cd8-bdcc9c6b99f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:12:01.586104 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecdf1e9-485f-4ec6-8cd8-bdcc9c6b99f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:12:01.586738 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecdf1e9-485f-4ec6-8cd8-bdcc9c6b99f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:12:01.587540 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecdf1e9-485f-4ec6-8cd8-bdcc9c6b99f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:12:01.589329 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecdf1e9-485f-4ec6-8cd8-bdcc9c6b99f4 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:12:01.604894 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1225/2436] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:12:01.606709 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25179b3-052b-446c-a1cc-596242c2cf37 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:12:01.618953 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25179b3-052b-446c-a1cc-596242c2cf37 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:12:01.625857 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b609e01-eadf-44cd-988e-61d762d64ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:12:01.627246 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b609e01-eadf-44cd-988e-61d762d64ab8 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:12:01.627920 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b609e01-eadf-44cd-988e-61d762d64ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:12:01.630618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b609e01-eadf-44cd-988e-61d762d64ab8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:12:01.631867 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b609e01-eadf-44cd-988e-61d762d64ab8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:12:01.634773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b609e01-eadf-44cd-988e-61d762d64ab8 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:12:01.651913 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1212/2437] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:12:01.657538 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecdf1e9-485f-4ec6-8cd8-bdcc9c6b99f4 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:12:01.669114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecdf1e9-485f-4ec6-8cd8-bdcc9c6b99f4 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:12:01.674542 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b9db7-bd6a-4889-8ba6-b20dc293bd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:12:01.675610 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b9db7-bd6a-4889-8ba6-b20dc293bd97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:12:01.675995 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b9db7-bd6a-4889-8ba6-b20dc293bd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:12:01.676347 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b9db7-bd6a-4889-8ba6-b20dc293bd97 admin admin] PATH_INFO: `/v3/projects/790b5019b9114120a51718d99317f7a1/users/305d1ce79a4f43c8b98e0967dab43da1/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:12:01.699054 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b9db7-bd6a-4889-8ba6-b20dc293bd97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=790b5019b9114120a51718d99317f7a1, user_id=305d1ce79a4f43c8b98e0967dab43da1, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:12:01.703285 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b9db7-bd6a-4889-8ba6-b20dc293bd97 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:12:01.726827 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1226/2438] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:12:01 2022] PUT /identity/v3/projects/790b5019b9114120a51718d99317f7a1/users/305d1ce79a4f43c8b98e0967dab43da1/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:12:01.734993 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e9352-0e7f-496b-8a21-3f560bc56b02 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:12:01.735469 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e9352-0e7f-496b-8a21-3f560bc56b02 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:12:01.735716 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e9352-0e7f-496b-8a21-3f560bc56b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:12:01.756967 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-902e9352-0e7f-496b-8a21-3f560bc56b02 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:12:01.758339 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-902e9352-0e7f-496b-8a21-3f560bc56b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:12:01.847602 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1213/2439] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:12:01.858222 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b9db7-bd6a-4889-8ba6-b20dc293bd97 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:12:01.860057 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-cb6b9db7-bd6a-4889-8ba6-b20dc293bd97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:12:01.895457 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:12:01.979709 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b9db7-bd6a-4889-8ba6-b20dc293bd97 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:12:01.986603 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dee335-9d00-4625-a580-9acdeb5b3532 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:12:01.987495 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dee335-9d00-4625-a580-9acdeb5b3532 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:12:01.987782 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dee335-9d00-4625-a580-9acdeb5b3532 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:12:01.988124 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dee335-9d00-4625-a580-9acdeb5b3532 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:12:02.006165 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dee335-9d00-4625-a580-9acdeb5b3532 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:12:02.007977 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dee335-9d00-4625-a580-9acdeb5b3532 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:12:02.023130 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1227/2440] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:12:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:12:07.558818 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db177955-f57b-4d28-9db7-c4d1786d0d06 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:12:07.559268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db177955-f57b-4d28-9db7-c4d1786d0d06 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:12:07.559480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db177955-f57b-4d28-9db7-c4d1786d0d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:12:07.642188 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-db177955-f57b-4d28-9db7-c4d1786d0d06 None None] MFA Rules not processed for user `305d1ce79a4f43c8b98e0967dab43da1`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:12:07.643684 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-db177955-f57b-4d28-9db7-c4d1786d0d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:12:07.655515 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:12:07.730249 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1214/2441] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:12:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:12:07.748244 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dee335-9d00-4625-a580-9acdeb5b3532 service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:12:07.750172 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-96dee335-9d00-4625-a580-9acdeb5b3532 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:12:07.856733 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dee335-9d00-4625-a580-9acdeb5b3532 service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:12:07.864566 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f066b40-fae2-4a46-8acf-5a4c39a0565e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:12:07.865326 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f066b40-fae2-4a46-8acf-5a4c39a0565e service nova] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:12:07.865987 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f066b40-fae2-4a46-8acf-5a4c39a0565e service nova] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:12:07.866265 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f066b40-fae2-4a46-8acf-5a4c39a0565e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:12:07.875622 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f066b40-fae2-4a46-8acf-5a4c39a0565e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:12:07.878826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f066b40-fae2-4a46-8acf-5a4c39a0565e service nova] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:12:07.896145 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1228/2442] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:12:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:12:09.055474 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c147a0-5874-4d66-a522-f60479206117 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:12:09.055950 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c147a0-5874-4d66-a522-f60479206117 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:12:09.056072 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c147a0-5874-4d66-a522-f60479206117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:12:09.076315 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-64c147a0-5874-4d66-a522-f60479206117 None None] MFA Rules not processed for user `3488a7d2258242c0859a105c1d162767`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:12:09.077568 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-64c147a0-5874-4d66-a522-f60479206117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:12:09.149044 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1215/2443] 10.222.0.39 () {58 vars in 967 bytes} [Fri Aug 5 20:12:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:12:09.204137 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f066b40-fae2-4a46-8acf-5a4c39a0565e service nova] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:12:09.205700 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-6f066b40-fae2-4a46-8acf-5a4c39a0565e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:12:09.283260 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f066b40-fae2-4a46-8acf-5a4c39a0565e service nova] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:12:09.289333 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af699f6e-b122-47ce-a3a5-4a0578ff44a8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91d3d43cb0bd4ca3b8e8f10e29cc421c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:12:09.290263 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af699f6e-b122-47ce-a3a5-4a0578ff44a8 service swift] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:12:09.290703 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af699f6e-b122-47ce-a3a5-4a0578ff44a8 service swift] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:12:09.291329 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af699f6e-b122-47ce-a3a5-4a0578ff44a8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:12:09.302365 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d7f5a-9030-448b-846a-0aa35ed79e2b None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:12:09.303114 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d7f5a-9030-448b-846a-0aa35ed79e2b None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:12:09.303751 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d7f5a-9030-448b-846a-0aa35ed79e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:12:09.305410 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af699f6e-b122-47ce-a3a5-4a0578ff44a8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:12:09.307026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af699f6e-b122-47ce-a3a5-4a0578ff44a8 service swift] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:12:09.317126 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1229/2444] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Aug 5 20:12:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 20:12:09.343939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-824d7f5a-9030-448b-846a-0aa35ed79e2b None None] MFA Rules not processed for user `305d1ce79a4f43c8b98e0967dab43da1`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:12:09.346485 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-824d7f5a-9030-448b-846a-0aa35ed79e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:12:09.363801 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1216/2445] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:12:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:12:16.924376 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af699f6e-b122-47ce-a3a5-4a0578ff44a8 service swift] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:12:16.926227 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-af699f6e-b122-47ce-a3a5-4a0578ff44a8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:12:16.945503 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af699f6e-b122-47ce-a3a5-4a0578ff44a8 service swift] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:12:16.955633 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d093f18-44e1-4b97-bc17-68d9f8b56953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:12:16.956330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d093f18-44e1-4b97-bc17-68d9f8b56953 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:12:16.956668 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d093f18-44e1-4b97-bc17-68d9f8b56953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:12:16.956954 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d093f18-44e1-4b97-bc17-68d9f8b56953 admin admin] PATH_INFO: `/v3/users/368838695ea74570b7e30c17a211b82b` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:12:16.958391 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d093f18-44e1-4b97-bc17-68d9f8b56953 admin admin] RBAC: Authorizing `identity:delete_user(user_id=368838695ea74570b7e30c17a211b82b)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:12:16.960411 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d093f18-44e1-4b97-bc17-68d9f8b56953 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:12:16.994465 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:12:17.024091 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5d093f18-44e1-4b97-bc17-68d9f8b56953 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '368838695ea74570b7e30c17a211b82b'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:12:17.038836 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5d093f18-44e1-4b97-bc17-68d9f8b56953 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '368838695ea74570b7e30c17a211b82b'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:12:17.048219 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5d093f18-44e1-4b97-bc17-68d9f8b56953 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '368838695ea74570b7e30c17a211b82b'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:12:17.063755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5d093f18-44e1-4b97-bc17-68d9f8b56953 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '368838695ea74570b7e30c17a211b82b'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:12:17.064562 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5d093f18-44e1-4b97-bc17-68d9f8b56953 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '368838695ea74570b7e30c17a211b82b'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:12:17.076374 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1230/2446] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:12:16 2022] DELETE /identity/v3/users/368838695ea74570b7e30c17a211b82b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:12:17.491341 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d7f5a-9030-448b-846a-0aa35ed79e2b None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:12:17.504452 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d7f5a-9030-448b-846a-0aa35ed79e2b None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:12:17.582397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69409272-7e0c-4be3-9afb-0254fcc59116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:12:17.583080 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69409272-7e0c-4be3-9afb-0254fcc59116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:12:17.583382 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69409272-7e0c-4be3-9afb-0254fcc59116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:12:17.583655 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69409272-7e0c-4be3-9afb-0254fcc59116 admin admin] PATH_INFO: `/v3/projects/fe26e033d1844e28849ed4de857a7958` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:12:17.584821 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69409272-7e0c-4be3-9afb-0254fcc59116 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe26e033d1844e28849ed4de857a7958)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:12:17.586738 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69409272-7e0c-4be3-9afb-0254fcc59116 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:12:17.634895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-69409272-7e0c-4be3-9afb-0254fcc59116 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe26e033d1844e28849ed4de857a7958', 'request_id': 'req-69409272-7e0c-4be3-9afb-0254fcc59116'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:12:17.638072 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:12:17.652946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-69409272-7e0c-4be3-9afb-0254fcc59116 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe26e033d1844e28849ed4de857a7958', 'request_id': 'req-69409272-7e0c-4be3-9afb-0254fcc59116'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:12:17.660679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-69409272-7e0c-4be3-9afb-0254fcc59116 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe26e033d1844e28849ed4de857a7958', 'request_id': 'req-69409272-7e0c-4be3-9afb-0254fcc59116'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:12:17.674015 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-69409272-7e0c-4be3-9afb-0254fcc59116 admin admin] The token cache is being invalidate because project fe26e033d1844e28849ed4de857a7958 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:12:17.674306 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-69409272-7e0c-4be3-9afb-0254fcc59116 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:12:17.674959 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-69409272-7e0c-4be3-9afb-0254fcc59116 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:12:17.676467 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1217/2447] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:12:17 2022] DELETE /identity/v3/projects/fe26e033d1844e28849ed4de857a7958 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:12:17.681157 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d093f18-44e1-4b97-bc17-68d9f8b56953 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:12:17.682622 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5d093f18-44e1-4b97-bc17-68d9f8b56953 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:12:17.696401 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d093f18-44e1-4b97-bc17-68d9f8b56953 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:12:17.753811 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba775952-02e0-4dc0-be1f-3b858150e149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:12:17.754718 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba775952-02e0-4dc0-be1f-3b858150e149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:12:17.754996 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba775952-02e0-4dc0-be1f-3b858150e149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:12:17.755267 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba775952-02e0-4dc0-be1f-3b858150e149 admin admin] PATH_INFO: `/v3/users/4955744df809448f814df2b6777cb9cb` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:12:17.756417 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba775952-02e0-4dc0-be1f-3b858150e149 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4955744df809448f814df2b6777cb9cb)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:12:17.758361 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba775952-02e0-4dc0-be1f-3b858150e149 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:12:17.806742 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ba775952-02e0-4dc0-be1f-3b858150e149 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4955744df809448f814df2b6777cb9cb'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:12:17.817773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ba775952-02e0-4dc0-be1f-3b858150e149 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4955744df809448f814df2b6777cb9cb'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:12:17.825334 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ba775952-02e0-4dc0-be1f-3b858150e149 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4955744df809448f814df2b6777cb9cb'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:12:17.838161 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ba775952-02e0-4dc0-be1f-3b858150e149 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4955744df809448f814df2b6777cb9cb'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:12:17.838919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ba775952-02e0-4dc0-be1f-3b858150e149 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4955744df809448f814df2b6777cb9cb'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:12:17.845836 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1231/2448] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:12:17 2022] DELETE /identity/v3/users/4955744df809448f814df2b6777cb9cb => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:12:18.017465 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69409272-7e0c-4be3-9afb-0254fcc59116 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:12:18.018924 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-69409272-7e0c-4be3-9afb-0254fcc59116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:12:18.036637 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69409272-7e0c-4be3-9afb-0254fcc59116 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:12:18.096613 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd051df-e745-49a7-97fa-e8bdac2b43d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:12:18.097243 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd051df-e745-49a7-97fa-e8bdac2b43d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:12:18.097522 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd051df-e745-49a7-97fa-e8bdac2b43d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:12:18.097820 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd051df-e745-49a7-97fa-e8bdac2b43d4 admin admin] PATH_INFO: `/v3/projects/f44e23eb59d04a8d96ac26782c467d99` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:12:18.099136 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd051df-e745-49a7-97fa-e8bdac2b43d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f44e23eb59d04a8d96ac26782c467d99)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:12:18.101059 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd051df-e745-49a7-97fa-e8bdac2b43d4 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:12:18.147988 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8bd051df-e745-49a7-97fa-e8bdac2b43d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f44e23eb59d04a8d96ac26782c467d99', 'request_id': 'req-8bd051df-e745-49a7-97fa-e8bdac2b43d4'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:12:18.155824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8bd051df-e745-49a7-97fa-e8bdac2b43d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f44e23eb59d04a8d96ac26782c467d99', 'request_id': 'req-8bd051df-e745-49a7-97fa-e8bdac2b43d4'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:12:18.161269 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8bd051df-e745-49a7-97fa-e8bdac2b43d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f44e23eb59d04a8d96ac26782c467d99', 'request_id': 'req-8bd051df-e745-49a7-97fa-e8bdac2b43d4'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:12:18.172714 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8bd051df-e745-49a7-97fa-e8bdac2b43d4 admin admin] The token cache is being invalidate because project f44e23eb59d04a8d96ac26782c467d99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:12:18.172995 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8bd051df-e745-49a7-97fa-e8bdac2b43d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:12:18.173617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8bd051df-e745-49a7-97fa-e8bdac2b43d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:12:18.175107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1218/2449] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:12:18 2022] DELETE /identity/v3/projects/f44e23eb59d04a8d96ac26782c467d99 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:12:27.841440 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:DELETE=16,keystone:SELECT=72,keystone:INSERT=2 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:12:28.168993 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:DELETE=10,keystone:SELECT=57,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:12:49.256871 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba775952-02e0-4dc0-be1f-3b858150e149 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:12:49.258375 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-ba775952-02e0-4dc0-be1f-3b858150e149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:12:49.332316 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba775952-02e0-4dc0-be1f-3b858150e149 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:12:49.400969 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e872d-76b7-4812-987f-2b8f011b26f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:12:49.401505 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e872d-76b7-4812-987f-2b8f011b26f8 service nova] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:12:49.401786 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e872d-76b7-4812-987f-2b8f011b26f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:12:49.402061 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e872d-76b7-4812-987f-2b8f011b26f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:12:49.403817 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-b86e872d-76b7-4812-987f-2b8f011b26f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:12:49.488175 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e872d-76b7-4812-987f-2b8f011b26f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:12:49.489808 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e872d-76b7-4812-987f-2b8f011b26f8 service nova] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:12:49.509570 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1232/2450] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:12:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:12:49.563374 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd051df-e745-49a7-97fa-e8bdac2b43d4 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:12:49.564793 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-8bd051df-e745-49a7-97fa-e8bdac2b43d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:12:49.644844 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd051df-e745-49a7-97fa-e8bdac2b43d4 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:12:49.648450 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a49ab7-dc69-47cc-85c2-a7915388982f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:12:49.649081 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a49ab7-dc69-47cc-85c2-a7915388982f service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:12:49.649358 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a49ab7-dc69-47cc-85c2-a7915388982f service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:12:49.649628 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a49ab7-dc69-47cc-85c2-a7915388982f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:12:49.651429 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-c6a49ab7-dc69-47cc-85c2-a7915388982f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:12:49.666564 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a49ab7-dc69-47cc-85c2-a7915388982f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:12:49.668011 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a49ab7-dc69-47cc-85c2-a7915388982f service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:12:49.680255 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1219/2451] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:12:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:12:57.533409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e872d-76b7-4812-987f-2b8f011b26f8 service nova] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:12:57.542884 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e872d-76b7-4812-987f-2b8f011b26f8 service nova] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:12:57.546418 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aff1b6-3d86-4334-8d17-cae5cc448a29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:12:57.547602 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aff1b6-3d86-4334-8d17-cae5cc448a29 service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:12:57.547908 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aff1b6-3d86-4334-8d17-cae5cc448a29 service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:12:57.548241 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aff1b6-3d86-4334-8d17-cae5cc448a29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:12:57.549941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-c6aff1b6-3d86-4334-8d17-cae5cc448a29 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:12:57.559595 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:12:57.564254 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aff1b6-3d86-4334-8d17-cae5cc448a29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:12:57.565709 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aff1b6-3d86-4334-8d17-cae5cc448a29 service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:12:57.577574 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1233/2452] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:12:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:12:59.675117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=35 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:13:05.844436 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d566e-40fe-4ebb-a98a-068ca3fd6b97 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:13:05.845246 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d566e-40fe-4ebb-a98a-068ca3fd6b97 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:13:05.845246 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d566e-40fe-4ebb-a98a-068ca3fd6b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:13:05.875489 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-aa3d566e-40fe-4ebb-a98a-068ca3fd6b97 None None] MFA Rules not processed for user `1afe13c8fbb1468b8ef3d0d0c671208d`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:13:05.877826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-aa3d566e-40fe-4ebb-a98a-068ca3fd6b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:13:05.885505 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1220/2453] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:13:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:13:07.571987 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:13:08.572197 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e7547-3d77-42b8-afe6-b1ae28bf53da None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:13:08.572888 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e7547-3d77-42b8-afe6-b1ae28bf53da None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:13:08.572888 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e7547-3d77-42b8-afe6-b1ae28bf53da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:13:08.594140 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-5b6e7547-3d77-42b8-afe6-b1ae28bf53da None None] MFA Rules not processed for user `b5d959f3af3745ed8e0bc4a67faf34f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:13:08.595395 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5b6e7547-3d77-42b8-afe6-b1ae28bf53da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:13:08.681018 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1234/2454] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:13:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:13:09.931686 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d566e-40fe-4ebb-a98a-068ca3fd6b97 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:13:09.934018 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-aa3d566e-40fe-4ebb-a98a-068ca3fd6b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:13:10.017646 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d566e-40fe-4ebb-a98a-068ca3fd6b97 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:13:10.022549 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c1bc3-a504-431f-9e73-25ff7605c59f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:13:10.023284 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c1bc3-a504-431f-9e73-25ff7605c59f service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:13:10.023568 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c1bc3-a504-431f-9e73-25ff7605c59f service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:13:10.023851 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c1bc3-a504-431f-9e73-25ff7605c59f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:13:10.026014 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-841c1bc3-a504-431f-9e73-25ff7605c59f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:13:10.037037 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841c1bc3-a504-431f-9e73-25ff7605c59f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:13:10.038397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841c1bc3-a504-431f-9e73-25ff7605c59f service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:13:10.048118 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1221/2455] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:13:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:13:18.671241 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=25 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:13:20.043035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=32 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:13:22.314357 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e7547-3d77-42b8-afe6-b1ae28bf53da None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:13:22.315771 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5b6e7547-3d77-42b8-afe6-b1ae28bf53da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:13:22.335205 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e7547-3d77-42b8-afe6-b1ae28bf53da None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:13:22.350437 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c61868-ce5e-494e-a173-f511c15f7f33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:13:22.351319 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c61868-ce5e-494e-a173-f511c15f7f33 service nova] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:13:22.351672 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c61868-ce5e-494e-a173-f511c15f7f33 service nova] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:13:22.352103 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c61868-ce5e-494e-a173-f511c15f7f33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:13:22.354659 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-37c61868-ce5e-494e-a173-f511c15f7f33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:13:22.368348 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c61868-ce5e-494e-a173-f511c15f7f33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:13:22.370155 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c61868-ce5e-494e-a173-f511c15f7f33 service nova] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:13:22.381273 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1235/2456] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:13:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:13:23.683633 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c1bc3-a504-431f-9e73-25ff7605c59f service neutron] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:13:23.692753 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c1bc3-a504-431f-9e73-25ff7605c59f service neutron] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:13:23.696359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011079d9-2442-4a71-b699-939932ededac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:13:23.696998 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011079d9-2442-4a71-b699-939932ededac service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:13:23.697281 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011079d9-2442-4a71-b699-939932ededac service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:13:23.697558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011079d9-2442-4a71-b699-939932ededac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:13:23.699292 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-011079d9-2442-4a71-b699-939932ededac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:13:23.766219 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011079d9-2442-4a71-b699-939932ededac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:13:23.768503 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011079d9-2442-4a71-b699-939932ededac service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:13:23.777879 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1222/2457] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:13:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:13:30.799574 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c61868-ce5e-494e-a173-f511c15f7f33 service nova] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:13:30.801896 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-37c61868-ce5e-494e-a173-f511c15f7f33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:13:30.816250 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c61868-ce5e-494e-a173-f511c15f7f33 service nova] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:13:30.822395 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2f033-0cdb-4fb6-9544-d8f08261e0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:13:30.823143 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2f033-0cdb-4fb6-9544-d8f08261e0de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:13:30.823540 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2f033-0cdb-4fb6-9544-d8f08261e0de admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:13:30.823914 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2f033-0cdb-4fb6-9544-d8f08261e0de admin admin] PATH_INFO: `/v3/users/0e022456457d41b8812058252e4a43f4` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:13:30.825237 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2f033-0cdb-4fb6-9544-d8f08261e0de admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e022456457d41b8812058252e4a43f4)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:13:30.827231 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2f033-0cdb-4fb6-9544-d8f08261e0de admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:13:30.914808 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-14f2f033-0cdb-4fb6-9544-d8f08261e0de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e022456457d41b8812058252e4a43f4'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:13:30.925094 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-14f2f033-0cdb-4fb6-9544-d8f08261e0de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e022456457d41b8812058252e4a43f4'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:13:30.935376 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-14f2f033-0cdb-4fb6-9544-d8f08261e0de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e022456457d41b8812058252e4a43f4'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:13:30.953720 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-14f2f033-0cdb-4fb6-9544-d8f08261e0de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e022456457d41b8812058252e4a43f4'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:13:30.957295 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-14f2f033-0cdb-4fb6-9544-d8f08261e0de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e022456457d41b8812058252e4a43f4'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:13:30.971591 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1236/2458] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:13:30 2022] DELETE /identity/v3/users/0e022456457d41b8812058252e4a43f4 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:13:31.195415 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011079d9-2442-4a71-b699-939932ededac service neutron] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:13:31.213366 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011079d9-2442-4a71-b699-939932ededac service neutron] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:13:31.317328 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2778081-5a9d-43f5-b3b2-a699b5245522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:13:31.318318 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2778081-5a9d-43f5-b3b2-a699b5245522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:13:31.320994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2778081-5a9d-43f5-b3b2-a699b5245522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:13:31.321852 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2778081-5a9d-43f5-b3b2-a699b5245522 admin admin] PATH_INFO: `/v3/projects/3f8a529448f14ef18a72de884f17d0f6` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:13:31.322800 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2778081-5a9d-43f5-b3b2-a699b5245522 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f8a529448f14ef18a72de884f17d0f6)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:13:31.325045 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2778081-5a9d-43f5-b3b2-a699b5245522 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:13:31.385251 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-f2778081-5a9d-43f5-b3b2-a699b5245522 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f8a529448f14ef18a72de884f17d0f6', 'request_id': 'req-f2778081-5a9d-43f5-b3b2-a699b5245522'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:13:31.394530 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-f2778081-5a9d-43f5-b3b2-a699b5245522 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f8a529448f14ef18a72de884f17d0f6', 'request_id': 'req-f2778081-5a9d-43f5-b3b2-a699b5245522'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:13:31.400398 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-f2778081-5a9d-43f5-b3b2-a699b5245522 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f8a529448f14ef18a72de884f17d0f6', 'request_id': 'req-f2778081-5a9d-43f5-b3b2-a699b5245522'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:13:31.416374 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-f2778081-5a9d-43f5-b3b2-a699b5245522 admin admin] The token cache is being invalidate because project 3f8a529448f14ef18a72de884f17d0f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:13:31.416737 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-f2778081-5a9d-43f5-b3b2-a699b5245522 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:13:31.417460 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-f2778081-5a9d-43f5-b3b2-a699b5245522 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:13:31.419075 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1223/2459] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:13:31 2022] DELETE /identity/v3/projects/3f8a529448f14ef18a72de884f17d0f6 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:13:31.423602 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2f033-0cdb-4fb6-9544-d8f08261e0de admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:13:31.425084 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-14f2f033-0cdb-4fb6-9544-d8f08261e0de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:13:31.447909 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2f033-0cdb-4fb6-9544-d8f08261e0de admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:13:31.518922 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a53fc64-7f32-4bdb-9188-fe6cef15198f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:13:31.519562 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53fc64-7f32-4bdb-9188-fe6cef15198f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:13:31.519837 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53fc64-7f32-4bdb-9188-fe6cef15198f admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:13:31.520114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53fc64-7f32-4bdb-9188-fe6cef15198f admin admin] PATH_INFO: `/v3/users/08ad93b79675421cb798c11e1ad9473b` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:13:31.521305 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a53fc64-7f32-4bdb-9188-fe6cef15198f admin admin] RBAC: Authorizing `identity:delete_user(user_id=08ad93b79675421cb798c11e1ad9473b)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:13:31.523396 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a53fc64-7f32-4bdb-9188-fe6cef15198f admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:13:31.571016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-9a53fc64-7f32-4bdb-9188-fe6cef15198f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08ad93b79675421cb798c11e1ad9473b'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:13:31.582760 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-9a53fc64-7f32-4bdb-9188-fe6cef15198f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08ad93b79675421cb798c11e1ad9473b'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:13:31.589980 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:13:31.592207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-9a53fc64-7f32-4bdb-9188-fe6cef15198f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08ad93b79675421cb798c11e1ad9473b'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:13:31.605873 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-9a53fc64-7f32-4bdb-9188-fe6cef15198f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08ad93b79675421cb798c11e1ad9473b'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:13:31.606584 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-9a53fc64-7f32-4bdb-9188-fe6cef15198f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08ad93b79675421cb798c11e1ad9473b'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:13:31.612908 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1237/2460] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:13:31 2022] DELETE /identity/v3/users/08ad93b79675421cb798c11e1ad9473b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:13:31.798951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2778081-5a9d-43f5-b3b2-a699b5245522 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:13:31.800455 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-f2778081-5a9d-43f5-b3b2-a699b5245522 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:13:31.817355 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2778081-5a9d-43f5-b3b2-a699b5245522 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:13:31.842692 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:13:31.887393 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce97d9-5921-4c50-9213-3a8efb74dd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:13:31.888033 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce97d9-5921-4c50-9213-3a8efb74dd64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:13:31.888311 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce97d9-5921-4c50-9213-3a8efb74dd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:13:31.889226 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce97d9-5921-4c50-9213-3a8efb74dd64 admin admin] PATH_INFO: `/v3/projects/ceb165f588e94eaa8f866b68dd08f3cb` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:13:31.890664 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fce97d9-5921-4c50-9213-3a8efb74dd64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceb165f588e94eaa8f866b68dd08f3cb)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:13:31.892649 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fce97d9-5921-4c50-9213-3a8efb74dd64 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:13:31.945872 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-4fce97d9-5921-4c50-9213-3a8efb74dd64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceb165f588e94eaa8f866b68dd08f3cb', 'request_id': 'req-4fce97d9-5921-4c50-9213-3a8efb74dd64'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:13:31.955462 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-4fce97d9-5921-4c50-9213-3a8efb74dd64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceb165f588e94eaa8f866b68dd08f3cb', 'request_id': 'req-4fce97d9-5921-4c50-9213-3a8efb74dd64'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:13:31.961926 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-4fce97d9-5921-4c50-9213-3a8efb74dd64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceb165f588e94eaa8f866b68dd08f3cb', 'request_id': 'req-4fce97d9-5921-4c50-9213-3a8efb74dd64'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:13:31.975159 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-4fce97d9-5921-4c50-9213-3a8efb74dd64 admin admin] The token cache is being invalidate because project ceb165f588e94eaa8f866b68dd08f3cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:13:31.975452 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-4fce97d9-5921-4c50-9213-3a8efb74dd64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:13:31.976111 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-4fce97d9-5921-4c50-9213-3a8efb74dd64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:13:31.977859 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1224/2461] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:13:31 2022] DELETE /identity/v3/projects/ceb165f588e94eaa8f866b68dd08f3cb => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:13:41.608944 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=8,keystone:DELETE=2 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:13:41.971071 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:13:55.745533 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a53fc64-7f32-4bdb-9188-fe6cef15198f admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:13:55.747090 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-9a53fc64-7f32-4bdb-9188-fe6cef15198f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:13:55.840021 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a53fc64-7f32-4bdb-9188-fe6cef15198f admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:13:55.904566 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150a115-7418-49dd-b4cf-106a3da2e331 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:13:55.905297 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150a115-7418-49dd-b4cf-106a3da2e331 service nova] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:13:55.905498 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150a115-7418-49dd-b4cf-106a3da2e331 service nova] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:13:55.905768 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150a115-7418-49dd-b4cf-106a3da2e331 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:13:55.907650 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-b150a115-7418-49dd-b4cf-106a3da2e331 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:13:55.920101 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b150a115-7418-49dd-b4cf-106a3da2e331 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:13:55.921720 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b150a115-7418-49dd-b4cf-106a3da2e331 service nova] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:13:55.933577 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1238/2462] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:13:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:13:58.289123 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18b89f-bcdd-40bf-88f2-45a49730a174 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:13:58.289951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18b89f-bcdd-40bf-88f2-45a49730a174 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:13:58.290235 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18b89f-bcdd-40bf-88f2-45a49730a174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:13:58.342582 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-5e18b89f-bcdd-40bf-88f2-45a49730a174 None None] MFA Rules not processed for user `3488a7d2258242c0859a105c1d162767`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:13:58.343798 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-5e18b89f-bcdd-40bf-88f2-45a49730a174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:13:58.412065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1225/2463] 10.222.0.39 () {58 vars in 1009 bytes} [Fri Aug 5 20:13:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:13:58.464744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150a115-7418-49dd-b4cf-106a3da2e331 service nova] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:13:58.466178 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-b150a115-7418-49dd-b4cf-106a3da2e331 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:13:58.556233 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150a115-7418-49dd-b4cf-106a3da2e331 service nova] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:13:58.561361 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc7933-eb9d-4ce6-9a79-97810332da1a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91d3d43cb0bd4ca3b8e8f10e29cc421c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:13:58.561973 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc7933-eb9d-4ce6-9a79-97810332da1a service swift] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:13:58.562253 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc7933-eb9d-4ce6-9a79-97810332da1a service swift] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:13:58.562527 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc7933-eb9d-4ce6-9a79-97810332da1a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:13:58.568067 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:13:58.572753 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc7933-eb9d-4ce6-9a79-97810332da1a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:13:58.574796 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc7933-eb9d-4ce6-9a79-97810332da1a service swift] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:13:58.583961 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1239/2464] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Aug 5 20:13:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 20:14:08.409691 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=40 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:14:08.580261 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:14:14.017543 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e18b89f-bcdd-40bf-88f2-45a49730a174 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:14:14.019053 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-5e18b89f-bcdd-40bf-88f2-45a49730a174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:14:14.090428 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e18b89f-bcdd-40bf-88f2-45a49730a174 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:14:14.095313 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2db34-83bc-4c15-bc14-4c290d6c35f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61e2c954f9a49abad9f3de17ee39f63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:14:14.095921 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2db34-83bc-4c15-bc14-4c290d6c35f0 service placement] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:14:14.096253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2db34-83bc-4c15-bc14-4c290d6c35f0 service placement] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:14:14.096750 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2db34-83bc-4c15-bc14-4c290d6c35f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:14:14.098875 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-9ca2db34-83bc-4c15-bc14-4c290d6c35f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:14:14.110148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca2db34-83bc-4c15-bc14-4c290d6c35f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:14:14.111866 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca2db34-83bc-4c15-bc14-4c290d6c35f0 service placement] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:14:14.126916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1226/2465] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:14:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:14:16.118705 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6194e-5b66-4c33-ad6b-b3fb07711ff3 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:14:16.119182 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6194e-5b66-4c33-ad6b-b3fb07711ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:14:16.119280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6194e-5b66-4c33-ad6b-b3fb07711ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:14:16.139280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-7bc6194e-5b66-4c33-ad6b-b3fb07711ff3 None None] MFA Rules not processed for user `3488a7d2258242c0859a105c1d162767`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:14:16.143820 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-7bc6194e-5b66-4c33-ad6b-b3fb07711ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:14:16.148222 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1240/2466] 10.222.0.39 () {58 vars in 1009 bytes} [Fri Aug 5 20:14:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:14:16.162449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2db34-83bc-4c15-bc14-4c290d6c35f0 service placement] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:14:16.173729 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2db34-83bc-4c15-bc14-4c290d6c35f0 service placement] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:14:16.180945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df016e34-28f4-43fe-9e12-b8907824668d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91d3d43cb0bd4ca3b8e8f10e29cc421c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:14:16.181564 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df016e34-28f4-43fe-9e12-b8907824668d service swift] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:14:16.181875 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df016e34-28f4-43fe-9e12-b8907824668d service swift] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:14:16.182272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df016e34-28f4-43fe-9e12-b8907824668d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:14:16.192687 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df016e34-28f4-43fe-9e12-b8907824668d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:14:16.194394 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df016e34-28f4-43fe-9e12-b8907824668d service swift] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:14:16.203010 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1227/2467] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Aug 5 20:14:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 20:14:26.134277 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:14:26.199509 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=38 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:14:36.622728 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877f134-8b66-4251-b970-2b8177bdb18d None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:14:36.623286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877f134-8b66-4251-b970-2b8177bdb18d None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:14:36.623286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877f134-8b66-4251-b970-2b8177bdb18d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:14:36.644993 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-e877f134-8b66-4251-b970-2b8177bdb18d None None] MFA Rules not processed for user `3488a7d2258242c0859a105c1d162767`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:14:36.646389 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e877f134-8b66-4251-b970-2b8177bdb18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:14:36.654393 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1241/2468] 10.222.0.39 () {58 vars in 967 bytes} [Fri Aug 5 20:14:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:14:36.667436 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df016e34-28f4-43fe-9e12-b8907824668d service swift] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:14:36.679291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df016e34-28f4-43fe-9e12-b8907824668d service swift] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:14:36.685563 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0fd3d7-ce5d-4cc9-9ee0-7b746fce0de5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91d3d43cb0bd4ca3b8e8f10e29cc421c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:14:36.686206 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fd3d7-ce5d-4cc9-9ee0-7b746fce0de5 service swift] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:14:36.686489 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fd3d7-ce5d-4cc9-9ee0-7b746fce0de5 service swift] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:14:36.686766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fd3d7-ce5d-4cc9-9ee0-7b746fce0de5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:14:36.697312 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0fd3d7-ce5d-4cc9-9ee0-7b746fce0de5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:14:36.698990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0fd3d7-ce5d-4cc9-9ee0-7b746fce0de5 service swift] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:14:36.708554 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1228/2469] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Aug 5 20:14:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 20:14:46.639005 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:14:46.704199 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=6 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:14.692875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e877f134-8b66-4251-b970-2b8177bdb18d None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:14.694329 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e877f134-8b66-4251-b970-2b8177bdb18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:14.751585 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e877f134-8b66-4251-b970-2b8177bdb18d None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:14.755287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3cb16a-2806-4494-b611-7d3ad5d97b98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:14.756031 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3cb16a-2806-4494-b611-7d3ad5d97b98 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:14.756357 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3cb16a-2806-4494-b611-7d3ad5d97b98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:14.756624 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3cb16a-2806-4494-b611-7d3ad5d97b98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:14.758511 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5e3cb16a-2806-4494-b611-7d3ad5d97b98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:14.820848 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3cb16a-2806-4494-b611-7d3ad5d97b98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:14.822325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3cb16a-2806-4494-b611-7d3ad5d97b98 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:14.832281 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1242/2470] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:15:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:15:17.668509 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0fd3d7-ce5d-4cc9-9ee0-7b746fce0de5 service swift] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:17.685116 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0fd3d7-ce5d-4cc9-9ee0-7b746fce0de5 service swift] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:17.691726 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d8719-c457-4bb5-b774-1e82bd351a14 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61e2c954f9a49abad9f3de17ee39f63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:17.692289 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d8719-c457-4bb5-b774-1e82bd351a14 service placement] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:17.692681 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d8719-c457-4bb5-b774-1e82bd351a14 service placement] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:17.692974 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d8719-c457-4bb5-b774-1e82bd351a14 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:17.696312 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-871d8719-c457-4bb5-b774-1e82bd351a14 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:17.712640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d8719-c457-4bb5-b774-1e82bd351a14 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:17.715105 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d8719-c457-4bb5-b774-1e82bd351a14 service placement] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:17.732006 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1229/2471] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:15:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:15:17.763585 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3cb16a-2806-4494-b611-7d3ad5d97b98 service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:17.765178 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5e3cb16a-2806-4494-b611-7d3ad5d97b98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:17.783728 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3cb16a-2806-4494-b611-7d3ad5d97b98 service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:17.789729 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5931c-ac78-49a4-bc4e-7a045416ab9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:17.790802 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5931c-ac78-49a4-bc4e-7a045416ab9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:17.791155 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5931c-ac78-49a4-bc4e-7a045416ab9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:17.791501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5931c-ac78-49a4-bc4e-7a045416ab9b admin admin] PATH_INFO: `/v3/users/1afe13c8fbb1468b8ef3d0d0c671208d` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:17.793105 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa5931c-ac78-49a4-bc4e-7a045416ab9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1afe13c8fbb1468b8ef3d0d0c671208d)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:17.795198 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa5931c-ac78-49a4-bc4e-7a045416ab9b admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:17.867022 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-0fa5931c-ac78-49a4-bc4e-7a045416ab9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1afe13c8fbb1468b8ef3d0d0c671208d'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:17.874521 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-0fa5931c-ac78-49a4-bc4e-7a045416ab9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1afe13c8fbb1468b8ef3d0d0c671208d'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:17.881642 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-0fa5931c-ac78-49a4-bc4e-7a045416ab9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1afe13c8fbb1468b8ef3d0d0c671208d'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:17.894627 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-0fa5931c-ac78-49a4-bc4e-7a045416ab9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1afe13c8fbb1468b8ef3d0d0c671208d'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:17.895273 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-0fa5931c-ac78-49a4-bc4e-7a045416ab9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1afe13c8fbb1468b8ef3d0d0c671208d'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:17.901256 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1243/2472] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:15:17 2022] DELETE /identity/v3/users/1afe13c8fbb1468b8ef3d0d0c671208d => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:15:18.078740 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d8719-c457-4bb5-b774-1e82bd351a14 service placement] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:18.090188 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d8719-c457-4bb5-b774-1e82bd351a14 service placement] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:18.153542 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af174a4-1540-4696-8df1-3ea97bc7f627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:18.154178 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af174a4-1540-4696-8df1-3ea97bc7f627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:18.154458 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af174a4-1540-4696-8df1-3ea97bc7f627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:18.154730 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af174a4-1540-4696-8df1-3ea97bc7f627 admin admin] PATH_INFO: `/v3/projects/5b265913f9bd46b9b36c18d567d0cf66` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:18.155872 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af174a4-1540-4696-8df1-3ea97bc7f627 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b265913f9bd46b9b36c18d567d0cf66)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:18.157779 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af174a4-1540-4696-8df1-3ea97bc7f627 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:18.204530 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-1af174a4-1540-4696-8df1-3ea97bc7f627 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b265913f9bd46b9b36c18d567d0cf66', 'request_id': 'req-1af174a4-1540-4696-8df1-3ea97bc7f627'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:18.211773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-1af174a4-1540-4696-8df1-3ea97bc7f627 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b265913f9bd46b9b36c18d567d0cf66', 'request_id': 'req-1af174a4-1540-4696-8df1-3ea97bc7f627'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:18.217886 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-1af174a4-1540-4696-8df1-3ea97bc7f627 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b265913f9bd46b9b36c18d567d0cf66', 'request_id': 'req-1af174a4-1540-4696-8df1-3ea97bc7f627'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:18.228695 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-1af174a4-1540-4696-8df1-3ea97bc7f627 admin admin] The token cache is being invalidate because project 5b265913f9bd46b9b36c18d567d0cf66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:15:18.228979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-1af174a4-1540-4696-8df1-3ea97bc7f627 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:18.229661 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-1af174a4-1540-4696-8df1-3ea97bc7f627 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:18.231494 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1230/2473] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:15:18 2022] DELETE /identity/v3/projects/5b265913f9bd46b9b36c18d567d0cf66 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:15:18.236444 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2278ee-a2b2-4353-b00e-929a78dee6be None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:18.236779 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2278ee-a2b2-4353-b00e-929a78dee6be None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:18.237082 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2278ee-a2b2-4353-b00e-929a78dee6be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:18.257634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-9a2278ee-a2b2-4353-b00e-929a78dee6be None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:15:18.258857 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-9a2278ee-a2b2-4353-b00e-929a78dee6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:18.279496 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:18.322132 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1244/2474] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:15:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:15:18.330922 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d37c9-4062-47e0-b5a6-65647a59a3db None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:18.331211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d37c9-4062-47e0-b5a6-65647a59a3db None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:18.331473 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d37c9-4062-47e0-b5a6-65647a59a3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:18.352471 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-7b5d37c9-4062-47e0-b5a6-65647a59a3db None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:15:18.353718 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-7b5d37c9-4062-47e0-b5a6-65647a59a3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:18.363722 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1231/2475] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:15:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:15:18.367954 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2278ee-a2b2-4353-b00e-929a78dee6be None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:18.378386 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2278ee-a2b2-4353-b00e-929a78dee6be None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:18.382952 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f9e5e-fff0-4f3f-ac63-68c519203439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:18.383548 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f9e5e-fff0-4f3f-ac63-68c519203439 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:18.383824 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f9e5e-fff0-4f3f-ac63-68c519203439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:18.384095 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f9e5e-fff0-4f3f-ac63-68c519203439 admin admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:18.384816 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238f9e5e-fff0-4f3f-ac63-68c519203439 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:15:18.385111 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238f9e5e-fff0-4f3f-ac63-68c519203439 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:18.387323 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238f9e5e-fff0-4f3f-ac63-68c519203439 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:18.399270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1245/2476] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:15:18.403721 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d37c9-4062-47e0-b5a6-65647a59a3db None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:18.413074 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d37c9-4062-47e0-b5a6-65647a59a3db None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:18.418370 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32437a6-9cd0-421e-a2e9-76be6b590509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:18.418974 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32437a6-9cd0-421e-a2e9-76be6b590509 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:18.419257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32437a6-9cd0-421e-a2e9-76be6b590509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:18.419527 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32437a6-9cd0-421e-a2e9-76be6b590509 admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:18.420195 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32437a6-9cd0-421e-a2e9-76be6b590509 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:18.422165 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32437a6-9cd0-421e-a2e9-76be6b590509 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:18.466070 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1232/2477] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:15:18 2022] POST /identity/v3/projects => generated 355 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:15:18.470358 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f9e5e-fff0-4f3f-ac63-68c519203439 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:18.481276 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f9e5e-fff0-4f3f-ac63-68c519203439 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:18.536676 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975dd944-4b0d-4983-99ad-cee47628c8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:18.537316 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975dd944-4b0d-4983-99ad-cee47628c8c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:18.537601 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975dd944-4b0d-4983-99ad-cee47628c8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:18.537880 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975dd944-4b0d-4983-99ad-cee47628c8c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:18.538562 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975dd944-4b0d-4983-99ad-cee47628c8c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:18.540472 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975dd944-4b0d-4983-99ad-cee47628c8c7 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:18.562607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1246/2478] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:15:18 2022] POST /identity/v3/users => generated 278 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:15:18.566783 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32437a6-9cd0-421e-a2e9-76be6b590509 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:18.576985 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32437a6-9cd0-421e-a2e9-76be6b590509 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:18.581561 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c012fc3-d00a-4827-a0b3-c3d708c741b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:18.582161 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c012fc3-d00a-4827-a0b3-c3d708c741b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:18.582442 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c012fc3-d00a-4827-a0b3-c3d708c741b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:18.582715 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c012fc3-d00a-4827-a0b3-c3d708c741b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:18.583401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c012fc3-d00a-4827-a0b3-c3d708c741b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:18.585023 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c012fc3-d00a-4827-a0b3-c3d708c741b7 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:18.595223 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1233/2479] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:15:18.599183 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975dd944-4b0d-4983-99ad-cee47628c8c7 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:18.609742 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975dd944-4b0d-4983-99ad-cee47628c8c7 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:18.614380 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570d0ba-395c-4a91-85fd-bab75e08f319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:18.614984 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570d0ba-395c-4a91-85fd-bab75e08f319 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:18.615275 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570d0ba-395c-4a91-85fd-bab75e08f319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:18.615548 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570d0ba-395c-4a91-85fd-bab75e08f319 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:18.616268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f570d0ba-395c-4a91-85fd-bab75e08f319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:18.617919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f570d0ba-395c-4a91-85fd-bab75e08f319 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:18.629357 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1247/2480] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:15:18.633542 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c012fc3-d00a-4827-a0b3-c3d708c741b7 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:18.644078 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c012fc3-d00a-4827-a0b3-c3d708c741b7 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:18.648865 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df37f6-e634-4098-99d7-1ddb08b9833b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:18.649581 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df37f6-e634-4098-99d7-1ddb08b9833b admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:18.649866 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df37f6-e634-4098-99d7-1ddb08b9833b admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:18.650144 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df37f6-e634-4098-99d7-1ddb08b9833b admin admin] PATH_INFO: `/v3/projects/4c1bb521078447c5a3e2f058eaf25d25/users/750d77af5e894de5a34a187f3726f78b/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:18.665221 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df37f6-e634-4098-99d7-1ddb08b9833b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c1bb521078447c5a3e2f058eaf25d25, user_id=750d77af5e894de5a34a187f3726f78b, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:18.668602 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df37f6-e634-4098-99d7-1ddb08b9833b admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:18.681317 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1234/2481] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:15:18 2022] PUT /identity/v3/projects/4c1bb521078447c5a3e2f058eaf25d25/users/750d77af5e894de5a34a187f3726f78b/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:15:18.692332 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8d7b1-07f8-4004-8afb-b60b3d6a623d None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:18.692655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8d7b1-07f8-4004-8afb-b60b3d6a623d None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:18.692958 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8d7b1-07f8-4004-8afb-b60b3d6a623d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:18.721064 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-03e8d7b1-07f8-4004-8afb-b60b3d6a623d None None] MFA Rules not processed for user `750d77af5e894de5a34a187f3726f78b`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:15:18.722188 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-03e8d7b1-07f8-4004-8afb-b60b3d6a623d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:18.792735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1248/2482] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:15:18 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:15:18.797371 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df37f6-e634-4098-99d7-1ddb08b9833b admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:18.803221 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:18.865480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df37f6-e634-4098-99d7-1ddb08b9833b admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:18.870170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaac17b-8dfa-452a-91e3-b929b8a29d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:18.871629 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaac17b-8dfa-452a-91e3-b929b8a29d22 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:18.871852 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaac17b-8dfa-452a-91e3-b929b8a29d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:18.872127 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaac17b-8dfa-452a-91e3-b929b8a29d22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:18.872919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaac17b-8dfa-452a-91e3-b929b8a29d22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:18.874823 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaac17b-8dfa-452a-91e3-b929b8a29d22 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:18.903707 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1235/2483] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:15:18 2022] POST /identity/v3/projects => generated 355 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:15:18.907827 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8d7b1-07f8-4004-8afb-b60b3d6a623d None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:18.920018 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8d7b1-07f8-4004-8afb-b60b3d6a623d None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:18.937468 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:18.979155 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65e3f1-533b-4edb-969a-e330c8c21ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:18.980336 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65e3f1-533b-4edb-969a-e330c8c21ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:18.980543 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65e3f1-533b-4edb-969a-e330c8c21ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:18.980836 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65e3f1-533b-4edb-969a-e330c8c21ce6 admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:18.981500 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65e3f1-533b-4edb-969a-e330c8c21ce6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:18.983416 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65e3f1-533b-4edb-969a-e330c8c21ce6 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:19.005304 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1249/2484] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:15:18 2022] POST /identity/v3/users => generated 284 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:15:19.009340 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaac17b-8dfa-452a-91e3-b929b8a29d22 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:19.020135 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaac17b-8dfa-452a-91e3-b929b8a29d22 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:19.024835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed06033-527d-4d28-9de4-20125c277e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:19.025470 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed06033-527d-4d28-9de4-20125c277e77 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:19.025753 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed06033-527d-4d28-9de4-20125c277e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:19.026057 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed06033-527d-4d28-9de4-20125c277e77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:19.027370 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed06033-527d-4d28-9de4-20125c277e77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:19.029129 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed06033-527d-4d28-9de4-20125c277e77 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:19.039645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1236/2485] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:15:19.044802 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65e3f1-533b-4edb-969a-e330c8c21ce6 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:19.055402 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65e3f1-533b-4edb-969a-e330c8c21ce6 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:19.060513 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119e0d8-aee3-4c43-bbff-41f2e4b33143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:19.061207 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119e0d8-aee3-4c43-bbff-41f2e4b33143 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:19.061489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119e0d8-aee3-4c43-bbff-41f2e4b33143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:19.061762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119e0d8-aee3-4c43-bbff-41f2e4b33143 admin admin] PATH_INFO: `/v3/projects/fc1d0778bed6401f8c1f2ca2e962c19d/users/d9f1620d1e0f487e9b2c015e007122b6/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:19.077285 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e119e0d8-aee3-4c43-bbff-41f2e4b33143 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc1d0778bed6401f8c1f2ca2e962c19d, user_id=d9f1620d1e0f487e9b2c015e007122b6, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:19.080043 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e119e0d8-aee3-4c43-bbff-41f2e4b33143 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:19.094074 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1250/2486] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:15:19 2022] PUT /identity/v3/projects/fc1d0778bed6401f8c1f2ca2e962c19d/users/d9f1620d1e0f487e9b2c015e007122b6/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:15:19.098143 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed06033-527d-4d28-9de4-20125c277e77 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:19.109949 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed06033-527d-4d28-9de4-20125c277e77 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:19.170730 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae5104-062a-46b0-bbb1-0bf50c784260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:19.171336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae5104-062a-46b0-bbb1-0bf50c784260 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:19.171614 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae5104-062a-46b0-bbb1-0bf50c784260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:19.171886 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae5104-062a-46b0-bbb1-0bf50c784260 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:19.172563 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edae5104-062a-46b0-bbb1-0bf50c784260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:19.174159 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edae5104-062a-46b0-bbb1-0bf50c784260 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:19.183915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1237/2487] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:15:19.188092 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119e0d8-aee3-4c43-bbff-41f2e4b33143 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:19.198438 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119e0d8-aee3-4c43-bbff-41f2e4b33143 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:19.203650 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38059803-c816-4dbc-86c8-14968bd00957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:19.204304 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38059803-c816-4dbc-86c8-14968bd00957 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:19.204606 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38059803-c816-4dbc-86c8-14968bd00957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:19.204885 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38059803-c816-4dbc-86c8-14968bd00957 admin admin] PATH_INFO: `/v3/projects/fc1d0778bed6401f8c1f2ca2e962c19d/users/d9f1620d1e0f487e9b2c015e007122b6/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:19.206810 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38059803-c816-4dbc-86c8-14968bd00957 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc1d0778bed6401f8c1f2ca2e962c19d, user_id=d9f1620d1e0f487e9b2c015e007122b6, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:19.209514 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38059803-c816-4dbc-86c8-14968bd00957 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:19.220844 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1251/2488] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:15:19 2022] PUT /identity/v3/projects/fc1d0778bed6401f8c1f2ca2e962c19d/users/d9f1620d1e0f487e9b2c015e007122b6/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:15:19.233408 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365fae0-7df4-45fa-afd6-0fee419cb163 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:19.233623 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365fae0-7df4-45fa-afd6-0fee419cb163 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:19.233901 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365fae0-7df4-45fa-afd6-0fee419cb163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:19.262794 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-e365fae0-7df4-45fa-afd6-0fee419cb163 None None] MFA Rules not processed for user `d9f1620d1e0f487e9b2c015e007122b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:15:19.264002 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-e365fae0-7df4-45fa-afd6-0fee419cb163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:19.331461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:19.335936 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1238/2489] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:15:19 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:15:19.347734 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38059803-c816-4dbc-86c8-14968bd00957 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:19.349505 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-38059803-c816-4dbc-86c8-14968bd00957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:19.409228 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38059803-c816-4dbc-86c8-14968bd00957 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:19.413119 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9589b6-771d-4beb-8a49-098804e58849 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:19.413913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9589b6-771d-4beb-8a49-098804e58849 service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:19.414186 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9589b6-771d-4beb-8a49-098804e58849 service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:19.414452 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9589b6-771d-4beb-8a49-098804e58849 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:19.424490 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9589b6-771d-4beb-8a49-098804e58849 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:19.425949 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9589b6-771d-4beb-8a49-098804e58849 service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:19.436001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1252/2490] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:15:19 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:15:19.517156 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e365fae0-7df4-45fa-afd6-0fee419cb163 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:19.528010 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e365fae0-7df4-45fa-afd6-0fee419cb163 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:19.531892 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f6f5f-e798-4aa0-b17e-1a7419d0fe6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:19.532492 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f6f5f-e798-4aa0-b17e-1a7419d0fe6d service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:19.532795 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f6f5f-e798-4aa0-b17e-1a7419d0fe6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:19.533064 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f6f5f-e798-4aa0-b17e-1a7419d0fe6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:19.588635 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f6f5f-e798-4aa0-b17e-1a7419d0fe6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:19.590700 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f6f5f-e798-4aa0-b17e-1a7419d0fe6d service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:19.599325 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1239/2491] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:15:19 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:15:29.431325 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=62,keystone:INSERT=5 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:29.595349 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=28 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:34.532124 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9589b6-771d-4beb-8a49-098804e58849 service cinder] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:34.591752 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9589b6-771d-4beb-8a49-098804e58849 service cinder] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:34.596452 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a00998-9974-42f3-9c94-8d0c4ec9ef15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:34.597099 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a00998-9974-42f3-9c94-8d0c4ec9ef15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:34.597377 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a00998-9974-42f3-9c94-8d0c4ec9ef15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:34.597648 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a00998-9974-42f3-9c94-8d0c4ec9ef15 admin admin] PATH_INFO: `/v3/users/750d77af5e894de5a34a187f3726f78b` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:34.598778 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a00998-9974-42f3-9c94-8d0c4ec9ef15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=750d77af5e894de5a34a187f3726f78b)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:34.600713 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a00998-9974-42f3-9c94-8d0c4ec9ef15 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:34.646075 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-77a00998-9974-42f3-9c94-8d0c4ec9ef15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '750d77af5e894de5a34a187f3726f78b'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:34.653689 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-77a00998-9974-42f3-9c94-8d0c4ec9ef15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '750d77af5e894de5a34a187f3726f78b'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:34.660635 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-77a00998-9974-42f3-9c94-8d0c4ec9ef15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '750d77af5e894de5a34a187f3726f78b'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:34.672294 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-77a00998-9974-42f3-9c94-8d0c4ec9ef15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '750d77af5e894de5a34a187f3726f78b'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:34.673132 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-77a00998-9974-42f3-9c94-8d0c4ec9ef15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '750d77af5e894de5a34a187f3726f78b'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:34.679113 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1253/2492] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:15:34 2022] DELETE /identity/v3/users/750d77af5e894de5a34a187f3726f78b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:15:34.683596 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd266dd-158e-4571-8570-1325a3cc09c8 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:34.684072 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd266dd-158e-4571-8570-1325a3cc09c8 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:34.684183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd266dd-158e-4571-8570-1325a3cc09c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:34.704789 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-4bd266dd-158e-4571-8570-1325a3cc09c8 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:15:34.705988 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-4bd266dd-158e-4571-8570-1325a3cc09c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:34.764522 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1240/2493] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:15:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:15:34.774977 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a00998-9974-42f3-9c94-8d0c4ec9ef15 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:34.776377 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-77a00998-9974-42f3-9c94-8d0c4ec9ef15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:34.835363 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a00998-9974-42f3-9c94-8d0c4ec9ef15 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:34.839498 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e410534d-ad48-4e3f-bca5-9a8ad205be23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:34.840139 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e410534d-ad48-4e3f-bca5-9a8ad205be23 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:34.840827 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e410534d-ad48-4e3f-bca5-9a8ad205be23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:34.840827 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e410534d-ad48-4e3f-bca5-9a8ad205be23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:34.851656 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e410534d-ad48-4e3f-bca5-9a8ad205be23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:34.853155 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e410534d-ad48-4e3f-bca5-9a8ad205be23 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:34.863561 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1254/2494] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:15:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:15:35.143886 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd266dd-158e-4571-8570-1325a3cc09c8 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:35.145576 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-4bd266dd-158e-4571-8570-1325a3cc09c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:35.158105 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd266dd-158e-4571-8570-1325a3cc09c8 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:35.162941 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a3af4-db3c-4d83-bf2f-659c95f427a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:35.163705 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a3af4-db3c-4d83-bf2f-659c95f427a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:35.163972 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a3af4-db3c-4d83-bf2f-659c95f427a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:35.164248 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a3af4-db3c-4d83-bf2f-659c95f427a4 admin admin] PATH_INFO: `/v3/projects/4c1bb521078447c5a3e2f058eaf25d25` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:35.165458 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a3af4-db3c-4d83-bf2f-659c95f427a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c1bb521078447c5a3e2f058eaf25d25)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:35.167458 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a3af4-db3c-4d83-bf2f-659c95f427a4 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:35.220603 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-601a3af4-db3c-4d83-bf2f-659c95f427a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c1bb521078447c5a3e2f058eaf25d25', 'request_id': 'req-601a3af4-db3c-4d83-bf2f-659c95f427a4'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:35.227742 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-601a3af4-db3c-4d83-bf2f-659c95f427a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c1bb521078447c5a3e2f058eaf25d25', 'request_id': 'req-601a3af4-db3c-4d83-bf2f-659c95f427a4'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:35.233198 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-601a3af4-db3c-4d83-bf2f-659c95f427a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c1bb521078447c5a3e2f058eaf25d25', 'request_id': 'req-601a3af4-db3c-4d83-bf2f-659c95f427a4'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:35.244905 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-601a3af4-db3c-4d83-bf2f-659c95f427a4 admin admin] The token cache is being invalidate because project 4c1bb521078447c5a3e2f058eaf25d25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:15:35.245188 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-601a3af4-db3c-4d83-bf2f-659c95f427a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:35.245814 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-601a3af4-db3c-4d83-bf2f-659c95f427a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:35.247400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1241/2495] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:15:35 2022] DELETE /identity/v3/projects/4c1bb521078447c5a3e2f058eaf25d25 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:15:35.252025 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e410534d-ad48-4e3f-bca5-9a8ad205be23 service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:35.263291 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e410534d-ad48-4e3f-bca5-9a8ad205be23 service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:35.283523 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:35.328105 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd81cb-5f94-4dd6-b6cf-4d4878e188c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:35.329122 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd81cb-5f94-4dd6-b6cf-4d4878e188c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:35.329448 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd81cb-5f94-4dd6-b6cf-4d4878e188c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:35.329915 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd81cb-5f94-4dd6-b6cf-4d4878e188c0 admin admin] PATH_INFO: `/v3/users/d9f1620d1e0f487e9b2c015e007122b6` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:35.331282 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd81cb-5f94-4dd6-b6cf-4d4878e188c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9f1620d1e0f487e9b2c015e007122b6)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:35.333293 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd81cb-5f94-4dd6-b6cf-4d4878e188c0 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:35.383261 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-01cd81cb-5f94-4dd6-b6cf-4d4878e188c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9f1620d1e0f487e9b2c015e007122b6'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:35.392201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-01cd81cb-5f94-4dd6-b6cf-4d4878e188c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9f1620d1e0f487e9b2c015e007122b6'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:35.400420 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-01cd81cb-5f94-4dd6-b6cf-4d4878e188c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9f1620d1e0f487e9b2c015e007122b6'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:35.413987 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-01cd81cb-5f94-4dd6-b6cf-4d4878e188c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9f1620d1e0f487e9b2c015e007122b6'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:35.414718 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-01cd81cb-5f94-4dd6-b6cf-4d4878e188c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9f1620d1e0f487e9b2c015e007122b6'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:35.420816 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1255/2496] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:15:35 2022] DELETE /identity/v3/users/d9f1620d1e0f487e9b2c015e007122b6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:15:35.486305 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a3af4-db3c-4d83-bf2f-659c95f427a4 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:35.489053 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-601a3af4-db3c-4d83-bf2f-659c95f427a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:35.557941 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a3af4-db3c-4d83-bf2f-659c95f427a4 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:35.601780 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:35.616046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb2d867-b0e5-4899-8166-8b5efda5866c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:35.616698 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2d867-b0e5-4899-8166-8b5efda5866c service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:35.616976 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2d867-b0e5-4899-8166-8b5efda5866c service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:35.617248 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2d867-b0e5-4899-8166-8b5efda5866c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:35.618974 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-6fb2d867-b0e5-4899-8166-8b5efda5866c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:35.681378 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb2d867-b0e5-4899-8166-8b5efda5866c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:35.682814 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb2d867-b0e5-4899-8166-8b5efda5866c service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:35.693628 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1242/2497] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:15:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:15:35.712785 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd81cb-5f94-4dd6-b6cf-4d4878e188c0 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:35.714383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-01cd81cb-5f94-4dd6-b6cf-4d4878e188c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:35.809243 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd81cb-5f94-4dd6-b6cf-4d4878e188c0 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:35.814520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80697ecb-b090-434c-b86e-8da16d865b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:35.815161 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80697ecb-b090-434c-b86e-8da16d865b69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:35.815440 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80697ecb-b090-434c-b86e-8da16d865b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:35.815716 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80697ecb-b090-434c-b86e-8da16d865b69 admin admin] PATH_INFO: `/v3/projects/fc1d0778bed6401f8c1f2ca2e962c19d` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:35.816900 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80697ecb-b090-434c-b86e-8da16d865b69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc1d0778bed6401f8c1f2ca2e962c19d)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:35.818772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80697ecb-b090-434c-b86e-8da16d865b69 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:35.871494 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:35.876737 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-80697ecb-b090-434c-b86e-8da16d865b69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc1d0778bed6401f8c1f2ca2e962c19d', 'request_id': 'req-80697ecb-b090-434c-b86e-8da16d865b69'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:35.885431 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-80697ecb-b090-434c-b86e-8da16d865b69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc1d0778bed6401f8c1f2ca2e962c19d', 'request_id': 'req-80697ecb-b090-434c-b86e-8da16d865b69'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:35.896571 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-80697ecb-b090-434c-b86e-8da16d865b69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc1d0778bed6401f8c1f2ca2e962c19d', 'request_id': 'req-80697ecb-b090-434c-b86e-8da16d865b69'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:35.908039 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-80697ecb-b090-434c-b86e-8da16d865b69 admin admin] The token cache is being invalidate because project fc1d0778bed6401f8c1f2ca2e962c19d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:15:35.908330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-80697ecb-b090-434c-b86e-8da16d865b69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:35.909015 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-80697ecb-b090-434c-b86e-8da16d865b69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:35.910529 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1256/2498] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:15:35 2022] DELETE /identity/v3/projects/fc1d0778bed6401f8c1f2ca2e962c19d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:15:35.915481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c945f83-f936-408b-a616-09c8ef20a998 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:35.915790 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c945f83-f936-408b-a616-09c8ef20a998 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:35.916088 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c945f83-f936-408b-a616-09c8ef20a998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:35.936974 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-7c945f83-f936-408b-a616-09c8ef20a998 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:15:35.938968 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-7c945f83-f936-408b-a616-09c8ef20a998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:35.998484 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1243/2499] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:15:36.006218 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376e643-b67d-41b5-9c9d-81bfc58d1f47 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:36.006498 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376e643-b67d-41b5-9c9d-81bfc58d1f47 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:36.006771 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376e643-b67d-41b5-9c9d-81bfc58d1f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:36.026708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-1376e643-b67d-41b5-9c9d-81bfc58d1f47 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:15:36.027894 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-1376e643-b67d-41b5-9c9d-81bfc58d1f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:36.036299 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1257/2500] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:15:36.040514 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c945f83-f936-408b-a616-09c8ef20a998 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:36.051551 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c945f83-f936-408b-a616-09c8ef20a998 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:36.056164 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e5ce2-0435-475e-b0cc-791251866f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:36.056829 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e5ce2-0435-475e-b0cc-791251866f6f admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:36.057090 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e5ce2-0435-475e-b0cc-791251866f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:36.057361 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e5ce2-0435-475e-b0cc-791251866f6f admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:36.058047 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e5ce2-0435-475e-b0cc-791251866f6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:15:36.058356 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e5ce2-0435-475e-b0cc-791251866f6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:36.059927 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e5ce2-0435-475e-b0cc-791251866f6f admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:36.071944 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1244/2501] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:15:36.076337 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376e643-b67d-41b5-9c9d-81bfc58d1f47 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:36.086431 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376e643-b67d-41b5-9c9d-81bfc58d1f47 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:36.091166 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842999df-8593-4234-a790-581e80438b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:36.091769 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842999df-8593-4234-a790-581e80438b88 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:36.092146 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842999df-8593-4234-a790-581e80438b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:36.092425 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842999df-8593-4234-a790-581e80438b88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:36.093187 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842999df-8593-4234-a790-581e80438b88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:36.095707 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842999df-8593-4234-a790-581e80438b88 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:36.130644 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1258/2502] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:15:36 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:15:36.134700 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e5ce2-0435-475e-b0cc-791251866f6f admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:36.145229 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e5ce2-0435-475e-b0cc-791251866f6f admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:36.201975 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fabfd10-379d-4f18-937f-635a33adab55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:36.202580 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabfd10-379d-4f18-937f-635a33adab55 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:36.202857 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabfd10-379d-4f18-937f-635a33adab55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:36.203129 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabfd10-379d-4f18-937f-635a33adab55 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:36.203838 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fabfd10-379d-4f18-937f-635a33adab55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:36.205910 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fabfd10-379d-4f18-937f-635a33adab55 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:36.227317 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1245/2503] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:15:36 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:15:36.231504 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842999df-8593-4234-a790-581e80438b88 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:36.242567 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842999df-8593-4234-a790-581e80438b88 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:36.247269 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9904fb5-4fb5-4a51-ac41-0b2ed165bd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:36.247874 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9904fb5-4fb5-4a51-ac41-0b2ed165bd53 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:36.248158 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9904fb5-4fb5-4a51-ac41-0b2ed165bd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:36.248436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9904fb5-4fb5-4a51-ac41-0b2ed165bd53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:36.249218 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9904fb5-4fb5-4a51-ac41-0b2ed165bd53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:36.250977 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9904fb5-4fb5-4a51-ac41-0b2ed165bd53 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:36.261118 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1259/2504] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:15:36.265248 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fabfd10-379d-4f18-937f-635a33adab55 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:36.277695 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fabfd10-379d-4f18-937f-635a33adab55 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:36.283510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30d517-e64b-4df6-8c5b-1b807e40d2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:36.284117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30d517-e64b-4df6-8c5b-1b807e40d2f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:36.284395 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30d517-e64b-4df6-8c5b-1b807e40d2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:36.284705 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30d517-e64b-4df6-8c5b-1b807e40d2f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:36.285391 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30d517-e64b-4df6-8c5b-1b807e40d2f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:36.286975 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30d517-e64b-4df6-8c5b-1b807e40d2f3 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:36.292919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:36.302951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1246/2505] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:15:36.304301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9904fb5-4fb5-4a51-ac41-0b2ed165bd53 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:36.314364 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9904fb5-4fb5-4a51-ac41-0b2ed165bd53 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:36.319047 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e73340-7041-4f7e-b2ce-31cb95a20918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:36.319705 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e73340-7041-4f7e-b2ce-31cb95a20918 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:36.319991 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e73340-7041-4f7e-b2ce-31cb95a20918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:36.320269 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e73340-7041-4f7e-b2ce-31cb95a20918 admin admin] PATH_INFO: `/v3/projects/9957a4f8aba44e95a8a9a06784302c14/users/b981341d0bd140e8b02ea2e93542d894/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:36.335083 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e73340-7041-4f7e-b2ce-31cb95a20918 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9957a4f8aba44e95a8a9a06784302c14, user_id=b981341d0bd140e8b02ea2e93542d894, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:36.337796 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e73340-7041-4f7e-b2ce-31cb95a20918 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:36.351237 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1260/2506] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:15:36 2022] PUT /identity/v3/projects/9957a4f8aba44e95a8a9a06784302c14/users/b981341d0bd140e8b02ea2e93542d894/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:15:36.363449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2878e4-a8a5-4f7e-9af2-a23aefdfc75c None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:36.363736 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2878e4-a8a5-4f7e-9af2-a23aefdfc75c None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:36.364013 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2878e4-a8a5-4f7e-9af2-a23aefdfc75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:36.392963 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-5f2878e4-a8a5-4f7e-9af2-a23aefdfc75c None None] MFA Rules not processed for user `b981341d0bd140e8b02ea2e93542d894`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:15:36.394427 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-5f2878e4-a8a5-4f7e-9af2-a23aefdfc75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:36.460520 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1247/2507] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:15:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:15:36.465353 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e73340-7041-4f7e-b2ce-31cb95a20918 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:36.527392 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e73340-7041-4f7e-b2ce-31cb95a20918 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:36.532628 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6777532-62a1-41ce-b978-db3e40f86073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:36.533245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6777532-62a1-41ce-b978-db3e40f86073 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:36.533531 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6777532-62a1-41ce-b978-db3e40f86073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:36.533810 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6777532-62a1-41ce-b978-db3e40f86073 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:36.534475 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6777532-62a1-41ce-b978-db3e40f86073 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:36.536398 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6777532-62a1-41ce-b978-db3e40f86073 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:36.563212 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1261/2508] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:15:36 2022] POST /identity/v3/projects => generated 367 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:15:36.567687 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2878e4-a8a5-4f7e-9af2-a23aefdfc75c None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:36.579105 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2878e4-a8a5-4f7e-9af2-a23aefdfc75c None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:36.636918 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c47595-1f78-4b0e-902f-477a8dbde78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:36.637528 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c47595-1f78-4b0e-902f-477a8dbde78b admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:36.637807 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c47595-1f78-4b0e-902f-477a8dbde78b admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:36.638080 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c47595-1f78-4b0e-902f-477a8dbde78b admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:36.638744 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c47595-1f78-4b0e-902f-477a8dbde78b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:36.640745 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c47595-1f78-4b0e-902f-477a8dbde78b admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:36.663793 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1248/2509] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:15:36 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:15:36.668439 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6777532-62a1-41ce-b978-db3e40f86073 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:36.684681 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6777532-62a1-41ce-b978-db3e40f86073 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:36.691805 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c943d21-fafb-4b15-a214-dab954f01444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:36.692623 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c943d21-fafb-4b15-a214-dab954f01444 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:36.692983 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c943d21-fafb-4b15-a214-dab954f01444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:36.693182 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c943d21-fafb-4b15-a214-dab954f01444 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:36.694073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c943d21-fafb-4b15-a214-dab954f01444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:36.695893 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c943d21-fafb-4b15-a214-dab954f01444 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:36.710140 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1262/2510] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:15:36.714513 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c47595-1f78-4b0e-902f-477a8dbde78b admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:36.725538 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c47595-1f78-4b0e-902f-477a8dbde78b admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:36.730175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ebb02-c871-4db0-a5d3-46d72f5abf24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:36.730837 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ebb02-c871-4db0-a5d3-46d72f5abf24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:36.731119 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ebb02-c871-4db0-a5d3-46d72f5abf24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:36.731399 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ebb02-c871-4db0-a5d3-46d72f5abf24 admin admin] PATH_INFO: `/v3/projects/ce77cc956e9e4a5e8c07134118b6d335/users/2392983b2507402eb0780e11299da2b1/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:36.746493 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ebb02-c871-4db0-a5d3-46d72f5abf24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce77cc956e9e4a5e8c07134118b6d335, user_id=2392983b2507402eb0780e11299da2b1, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:36.749196 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ebb02-c871-4db0-a5d3-46d72f5abf24 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:36.770983 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1249/2511] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:15:36 2022] PUT /identity/v3/projects/ce77cc956e9e4a5e8c07134118b6d335/users/2392983b2507402eb0780e11299da2b1/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:15:36.775164 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c943d21-fafb-4b15-a214-dab954f01444 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:36.786037 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c943d21-fafb-4b15-a214-dab954f01444 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:36.825945 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:36.845184 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e93e2-d8a1-4e6a-9677-b720540b800a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:36.845739 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e93e2-d8a1-4e6a-9677-b720540b800a admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:36.846026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e93e2-d8a1-4e6a-9677-b720540b800a admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:36.846310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e93e2-d8a1-4e6a-9677-b720540b800a admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:36.847015 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e93e2-d8a1-4e6a-9677-b720540b800a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:36.848696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e93e2-d8a1-4e6a-9677-b720540b800a admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:36.865302 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1263/2512] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:15:36.869472 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ebb02-c871-4db0-a5d3-46d72f5abf24 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:36.880282 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ebb02-c871-4db0-a5d3-46d72f5abf24 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:36.885308 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc347f2-8313-4651-814e-765e75872ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:36.885972 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc347f2-8313-4651-814e-765e75872ca9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:36.886260 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc347f2-8313-4651-814e-765e75872ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:36.886539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc347f2-8313-4651-814e-765e75872ca9 admin admin] PATH_INFO: `/v3/projects/ce77cc956e9e4a5e8c07134118b6d335/users/2392983b2507402eb0780e11299da2b1/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:36.888544 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc347f2-8313-4651-814e-765e75872ca9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce77cc956e9e4a5e8c07134118b6d335, user_id=2392983b2507402eb0780e11299da2b1, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:36.891887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc347f2-8313-4651-814e-765e75872ca9 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:36.903340 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1250/2513] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:15:36 2022] PUT /identity/v3/projects/ce77cc956e9e4a5e8c07134118b6d335/users/2392983b2507402eb0780e11299da2b1/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:15:36.913901 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de7317-7b34-423c-90aa-beacce88acf3 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:36.914191 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de7317-7b34-423c-90aa-beacce88acf3 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:36.914467 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de7317-7b34-423c-90aa-beacce88acf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:36.945151 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-28de7317-7b34-423c-90aa-beacce88acf3 None None] MFA Rules not processed for user `2392983b2507402eb0780e11299da2b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:15:36.947436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-28de7317-7b34-423c-90aa-beacce88acf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:37.020954 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1264/2514] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:15:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:15:37.032613 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc347f2-8313-4651-814e-765e75872ca9 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:37.034381 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-adc347f2-8313-4651-814e-765e75872ca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:37.100025 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:37.113813 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc347f2-8313-4651-814e-765e75872ca9 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:37.117465 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851473a6-c1dc-44f6-9320-e5f3c918dd2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:37.118106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851473a6-c1dc-44f6-9320-e5f3c918dd2d service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:37.118384 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851473a6-c1dc-44f6-9320-e5f3c918dd2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:37.118654 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851473a6-c1dc-44f6-9320-e5f3c918dd2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:37.128861 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851473a6-c1dc-44f6-9320-e5f3c918dd2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:37.130302 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851473a6-c1dc-44f6-9320-e5f3c918dd2d service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:37.140160 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1251/2515] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:15:37 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:15:37.194457 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de7317-7b34-423c-90aa-beacce88acf3 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:37.197541 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-28de7317-7b34-423c-90aa-beacce88acf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:37.213137 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de7317-7b34-423c-90aa-beacce88acf3 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:37.217461 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e49c3-b250-4cf4-abf5-10a35b93d00a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:37.217461 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e49c3-b250-4cf4-abf5-10a35b93d00a service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:37.217745 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e49c3-b250-4cf4-abf5-10a35b93d00a service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:37.218539 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e49c3-b250-4cf4-abf5-10a35b93d00a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:37.298298 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e49c3-b250-4cf4-abf5-10a35b93d00a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:37.299883 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e49c3-b250-4cf4-abf5-10a35b93d00a service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:37.309516 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1265/2516] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:15:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:15:40.677430 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851473a6-c1dc-44f6-9320-e5f3c918dd2d service cinder] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:40.752384 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851473a6-c1dc-44f6-9320-e5f3c918dd2d service cinder] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:40.757364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54b3f7-4b20-4f8b-8f3a-eabfa04f86f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:40.758183 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54b3f7-4b20-4f8b-8f3a-eabfa04f86f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:40.758461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54b3f7-4b20-4f8b-8f3a-eabfa04f86f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:40.758906 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54b3f7-4b20-4f8b-8f3a-eabfa04f86f9 admin admin] PATH_INFO: `/v3/users/b981341d0bd140e8b02ea2e93542d894` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:40.760101 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54b3f7-4b20-4f8b-8f3a-eabfa04f86f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b981341d0bd140e8b02ea2e93542d894)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:40.763153 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54b3f7-4b20-4f8b-8f3a-eabfa04f86f9 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:40.813389 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-1c54b3f7-4b20-4f8b-8f3a-eabfa04f86f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b981341d0bd140e8b02ea2e93542d894'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:40.822006 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-1c54b3f7-4b20-4f8b-8f3a-eabfa04f86f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b981341d0bd140e8b02ea2e93542d894'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:40.834241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-1c54b3f7-4b20-4f8b-8f3a-eabfa04f86f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b981341d0bd140e8b02ea2e93542d894'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:40.834878 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-1c54b3f7-4b20-4f8b-8f3a-eabfa04f86f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b981341d0bd140e8b02ea2e93542d894'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:40.841742 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-1c54b3f7-4b20-4f8b-8f3a-eabfa04f86f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b981341d0bd140e8b02ea2e93542d894'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:40.848299 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1252/2517] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:15:40 2022] DELETE /identity/v3/users/b981341d0bd140e8b02ea2e93542d894 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:15:40.853041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7af3b4-1578-4f7c-a1a8-d6ddcc6f7321 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:40.853554 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7af3b4-1578-4f7c-a1a8-d6ddcc6f7321 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:40.853668 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7af3b4-1578-4f7c-a1a8-d6ddcc6f7321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:40.873060 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-3e7af3b4-1578-4f7c-a1a8-d6ddcc6f7321 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:15:40.874245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-3e7af3b4-1578-4f7c-a1a8-d6ddcc6f7321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:40.939106 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1266/2518] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:15:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:15:40.948983 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54b3f7-4b20-4f8b-8f3a-eabfa04f86f9 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:40.950367 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-1c54b3f7-4b20-4f8b-8f3a-eabfa04f86f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:41.015150 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54b3f7-4b20-4f8b-8f3a-eabfa04f86f9 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:41.022283 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ce10c-bc44-4843-9084-ffbf23f79b4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:41.023356 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ce10c-bc44-4843-9084-ffbf23f79b4f service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:41.023861 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ce10c-bc44-4843-9084-ffbf23f79b4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:41.024351 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ce10c-bc44-4843-9084-ffbf23f79b4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:41.039261 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600ce10c-bc44-4843-9084-ffbf23f79b4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:41.040944 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600ce10c-bc44-4843-9084-ffbf23f79b4f service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:41.054273 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1253/2519] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:15:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:15:41.367691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7af3b4-1578-4f7c-a1a8-d6ddcc6f7321 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:41.369529 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-3e7af3b4-1578-4f7c-a1a8-d6ddcc6f7321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:41.379118 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:41.388152 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7af3b4-1578-4f7c-a1a8-d6ddcc6f7321 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:41.394294 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e8a7b5-9473-4705-a12d-be293f51a803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:41.395162 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e8a7b5-9473-4705-a12d-be293f51a803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:41.395607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e8a7b5-9473-4705-a12d-be293f51a803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:41.396031 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e8a7b5-9473-4705-a12d-be293f51a803 admin admin] PATH_INFO: `/v3/projects/9957a4f8aba44e95a8a9a06784302c14` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:41.397597 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e8a7b5-9473-4705-a12d-be293f51a803 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9957a4f8aba44e95a8a9a06784302c14)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:41.399823 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e8a7b5-9473-4705-a12d-be293f51a803 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:41.501454 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-a2e8a7b5-9473-4705-a12d-be293f51a803 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9957a4f8aba44e95a8a9a06784302c14', 'request_id': 'req-a2e8a7b5-9473-4705-a12d-be293f51a803'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:41.501454 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-a2e8a7b5-9473-4705-a12d-be293f51a803 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9957a4f8aba44e95a8a9a06784302c14', 'request_id': 'req-a2e8a7b5-9473-4705-a12d-be293f51a803'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:41.508677 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-a2e8a7b5-9473-4705-a12d-be293f51a803 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9957a4f8aba44e95a8a9a06784302c14', 'request_id': 'req-a2e8a7b5-9473-4705-a12d-be293f51a803'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:41.521576 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-a2e8a7b5-9473-4705-a12d-be293f51a803 admin admin] The token cache is being invalidate because project 9957a4f8aba44e95a8a9a06784302c14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:15:41.521940 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-a2e8a7b5-9473-4705-a12d-be293f51a803 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:41.522666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-a2e8a7b5-9473-4705-a12d-be293f51a803 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:41.524496 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1267/2520] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:15:41 2022] DELETE /identity/v3/projects/9957a4f8aba44e95a8a9a06784302c14 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:15:41.528664 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ce10c-bc44-4843-9084-ffbf23f79b4f service neutron] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:41.539158 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ce10c-bc44-4843-9084-ffbf23f79b4f service neutron] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:41.543580 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:41.604489 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a5e35-1bb1-4ba4-ba82-f370f118ed92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:41.605172 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a5e35-1bb1-4ba4-ba82-f370f118ed92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:41.605476 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a5e35-1bb1-4ba4-ba82-f370f118ed92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:41.605743 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a5e35-1bb1-4ba4-ba82-f370f118ed92 admin admin] PATH_INFO: `/v3/users/2392983b2507402eb0780e11299da2b1` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:41.606922 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a5e35-1bb1-4ba4-ba82-f370f118ed92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2392983b2507402eb0780e11299da2b1)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:41.608863 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a5e35-1bb1-4ba4-ba82-f370f118ed92 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:41.656000 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-384a5e35-1bb1-4ba4-ba82-f370f118ed92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2392983b2507402eb0780e11299da2b1'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:41.664245 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-384a5e35-1bb1-4ba4-ba82-f370f118ed92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2392983b2507402eb0780e11299da2b1'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:41.676835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-384a5e35-1bb1-4ba4-ba82-f370f118ed92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2392983b2507402eb0780e11299da2b1'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:41.677494 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-384a5e35-1bb1-4ba4-ba82-f370f118ed92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2392983b2507402eb0780e11299da2b1'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:41.684588 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-384a5e35-1bb1-4ba4-ba82-f370f118ed92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2392983b2507402eb0780e11299da2b1'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:41.691042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1254/2521] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:15:41 2022] DELETE /identity/v3/users/2392983b2507402eb0780e11299da2b1 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:15:41.969852 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e8a7b5-9473-4705-a12d-be293f51a803 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:41.972273 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-a2e8a7b5-9473-4705-a12d-be293f51a803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:41.988470 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e8a7b5-9473-4705-a12d-be293f51a803 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:42.067047 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcbd7ec-c0d4-442c-a723-ccb9197d9585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:42.068112 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbd7ec-c0d4-442c-a723-ccb9197d9585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:42.068662 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbd7ec-c0d4-442c-a723-ccb9197d9585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:42.069163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbd7ec-c0d4-442c-a723-ccb9197d9585 admin admin] PATH_INFO: `/v3/projects/ce77cc956e9e4a5e8c07134118b6d335` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:42.071175 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcbd7ec-c0d4-442c-a723-ccb9197d9585 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce77cc956e9e4a5e8c07134118b6d335)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:42.074914 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcbd7ec-c0d4-442c-a723-ccb9197d9585 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:42.123169 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5dcbd7ec-c0d4-442c-a723-ccb9197d9585 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce77cc956e9e4a5e8c07134118b6d335', 'request_id': 'req-5dcbd7ec-c0d4-442c-a723-ccb9197d9585'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:42.130242 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5dcbd7ec-c0d4-442c-a723-ccb9197d9585 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce77cc956e9e4a5e8c07134118b6d335', 'request_id': 'req-5dcbd7ec-c0d4-442c-a723-ccb9197d9585'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:42.138319 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5dcbd7ec-c0d4-442c-a723-ccb9197d9585 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce77cc956e9e4a5e8c07134118b6d335', 'request_id': 'req-5dcbd7ec-c0d4-442c-a723-ccb9197d9585'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:42.150993 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5dcbd7ec-c0d4-442c-a723-ccb9197d9585 admin admin] The token cache is being invalidate because project ce77cc956e9e4a5e8c07134118b6d335 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:15:42.151285 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5dcbd7ec-c0d4-442c-a723-ccb9197d9585 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:42.151951 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5dcbd7ec-c0d4-442c-a723-ccb9197d9585 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:42.153504 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1268/2522] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:15:41 2022] DELETE /identity/v3/projects/ce77cc956e9e4a5e8c07134118b6d335 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:15:42.159991 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c3eb4-2be5-40c9-bf16-b6514321365b None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:42.160405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c3eb4-2be5-40c9-bf16-b6514321365b None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:42.160819 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c3eb4-2be5-40c9-bf16-b6514321365b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:42.192037 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-255c3eb4-2be5-40c9-bf16-b6514321365b None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:15:42.193434 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-255c3eb4-2be5-40c9-bf16-b6514321365b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:42.253583 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1255/2523] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:15:42.261525 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05627d0d-e456-432a-a6eb-69158b481e05 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:42.261816 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05627d0d-e456-432a-a6eb-69158b481e05 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:42.262096 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05627d0d-e456-432a-a6eb-69158b481e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:42.282726 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-05627d0d-e456-432a-a6eb-69158b481e05 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:15:42.283932 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-05627d0d-e456-432a-a6eb-69158b481e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:42.292097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1269/2524] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:15:42.296991 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c3eb4-2be5-40c9-bf16-b6514321365b None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:42.309155 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c3eb4-2be5-40c9-bf16-b6514321365b None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:42.313936 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d1ec6-71ad-402c-affb-ca2507025d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:42.314543 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d1ec6-71ad-402c-affb-ca2507025d64 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:42.314824 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d1ec6-71ad-402c-affb-ca2507025d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:42.315097 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d1ec6-71ad-402c-affb-ca2507025d64 admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:42.315794 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d1ec6-71ad-402c-affb-ca2507025d64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:15:42.316101 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d1ec6-71ad-402c-affb-ca2507025d64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:42.317787 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d1ec6-71ad-402c-affb-ca2507025d64 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:42.333971 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1256/2525] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:15:42.339562 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05627d0d-e456-432a-a6eb-69158b481e05 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:42.350029 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05627d0d-e456-432a-a6eb-69158b481e05 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:42.354779 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3630271-f419-4c90-a702-81589e4ff56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:42.355412 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3630271-f419-4c90-a702-81589e4ff56b admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:42.355691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3630271-f419-4c90-a702-81589e4ff56b admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:42.355966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3630271-f419-4c90-a702-81589e4ff56b admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:42.356656 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3630271-f419-4c90-a702-81589e4ff56b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:42.358565 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3630271-f419-4c90-a702-81589e4ff56b admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:42.364459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:42.389435 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1270/2526] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:15:42 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:15:42.395955 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d1ec6-71ad-402c-affb-ca2507025d64 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:42.405969 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d1ec6-71ad-402c-affb-ca2507025d64 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:42.415735 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:42.465861 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea316d-594e-4aba-a353-139e468906bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:42.466475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea316d-594e-4aba-a353-139e468906bb admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:42.466756 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea316d-594e-4aba-a353-139e468906bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:42.467028 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea316d-594e-4aba-a353-139e468906bb admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:42.467687 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea316d-594e-4aba-a353-139e468906bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:42.470605 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea316d-594e-4aba-a353-139e468906bb admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:42.492288 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1257/2527] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:15:42 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:15:42.496440 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3630271-f419-4c90-a702-81589e4ff56b admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:42.506750 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3630271-f419-4c90-a702-81589e4ff56b admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:42.512179 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b3099-ecb9-476f-8be8-d1ada0b66821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:42.512836 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b3099-ecb9-476f-8be8-d1ada0b66821 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:42.513142 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b3099-ecb9-476f-8be8-d1ada0b66821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:42.513496 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b3099-ecb9-476f-8be8-d1ada0b66821 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:42.514099 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b3099-ecb9-476f-8be8-d1ada0b66821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:42.515729 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b3099-ecb9-476f-8be8-d1ada0b66821 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:42.528656 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1271/2528] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:15:42.533742 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea316d-594e-4aba-a353-139e468906bb admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:42.544195 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea316d-594e-4aba-a353-139e468906bb admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:42.557397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5349177-6266-492c-a92a-376f5ee89689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:42.557397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5349177-6266-492c-a92a-376f5ee89689 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:42.557397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5349177-6266-492c-a92a-376f5ee89689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:42.557397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5349177-6266-492c-a92a-376f5ee89689 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:42.557992 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5349177-6266-492c-a92a-376f5ee89689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:42.557992 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5349177-6266-492c-a92a-376f5ee89689 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:42.566375 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1258/2529] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:15:42.572535 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b3099-ecb9-476f-8be8-d1ada0b66821 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:42.584358 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b3099-ecb9-476f-8be8-d1ada0b66821 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:42.589128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ebb98-21a1-4561-a00d-4167dc2328a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:42.589785 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ebb98-21a1-4561-a00d-4167dc2328a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:42.590069 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ebb98-21a1-4561-a00d-4167dc2328a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:42.590344 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ebb98-21a1-4561-a00d-4167dc2328a1 admin admin] PATH_INFO: `/v3/projects/c9d01de6e09140ed9bc350fd0a4ea542/users/64555def954448b2bd8fde44d11f7242/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:42.613256 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ebb98-21a1-4561-a00d-4167dc2328a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9d01de6e09140ed9bc350fd0a4ea542, user_id=64555def954448b2bd8fde44d11f7242, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:42.615915 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ebb98-21a1-4561-a00d-4167dc2328a1 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:42.631266 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1272/2530] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:15:42 2022] PUT /identity/v3/projects/c9d01de6e09140ed9bc350fd0a4ea542/users/64555def954448b2bd8fde44d11f7242/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:15:42.643243 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eec71f-d591-4ecf-b874-f873ab58a8e2 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:42.643512 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eec71f-d591-4ecf-b874-f873ab58a8e2 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:42.644030 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eec71f-d591-4ecf-b874-f873ab58a8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:42.682953 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-d4eec71f-d591-4ecf-b874-f873ab58a8e2 None None] MFA Rules not processed for user `64555def954448b2bd8fde44d11f7242`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:15:42.684169 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-d4eec71f-d591-4ecf-b874-f873ab58a8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:42.757132 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1259/2531] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:15:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:15:42.762076 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ebb98-21a1-4561-a00d-4167dc2328a1 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:42.841083 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ebb98-21a1-4561-a00d-4167dc2328a1 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:42.848520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa7bf12-66f4-4ceb-97ec-5e9687cf5f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:42.849549 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa7bf12-66f4-4ceb-97ec-5e9687cf5f06 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:42.850209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa7bf12-66f4-4ceb-97ec-5e9687cf5f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:42.850652 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa7bf12-66f4-4ceb-97ec-5e9687cf5f06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:42.851786 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa7bf12-66f4-4ceb-97ec-5e9687cf5f06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:42.855241 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa7bf12-66f4-4ceb-97ec-5e9687cf5f06 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:42.897954 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1273/2532] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:15:42 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:15:42.902213 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eec71f-d591-4ecf-b874-f873ab58a8e2 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:42.921571 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eec71f-d591-4ecf-b874-f873ab58a8e2 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:43.008385 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254104ff-7e93-47e2-b5ce-7d43f86371d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:43.009543 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254104ff-7e93-47e2-b5ce-7d43f86371d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:43.010045 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254104ff-7e93-47e2-b5ce-7d43f86371d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:43.010535 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254104ff-7e93-47e2-b5ce-7d43f86371d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:43.011788 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254104ff-7e93-47e2-b5ce-7d43f86371d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:43.015765 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254104ff-7e93-47e2-b5ce-7d43f86371d1 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:43.060734 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1260/2533] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:15:42 2022] POST /identity/v3/users => generated 299 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:15:43.062097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa7bf12-66f4-4ceb-97ec-5e9687cf5f06 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:43.074882 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa7bf12-66f4-4ceb-97ec-5e9687cf5f06 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:43.081058 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e703b-7782-44ae-8bde-1b1a63576d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:43.082353 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e703b-7782-44ae-8bde-1b1a63576d48 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:43.082624 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e703b-7782-44ae-8bde-1b1a63576d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:43.083144 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e703b-7782-44ae-8bde-1b1a63576d48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:43.083953 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e703b-7782-44ae-8bde-1b1a63576d48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:43.085783 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e703b-7782-44ae-8bde-1b1a63576d48 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:43.097877 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1274/2534] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:15:43.103150 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254104ff-7e93-47e2-b5ce-7d43f86371d1 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:43.115215 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254104ff-7e93-47e2-b5ce-7d43f86371d1 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:43.120061 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393fc9a3-7523-4262-b6ae-981a858b8493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:43.120800 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393fc9a3-7523-4262-b6ae-981a858b8493 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:43.121077 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393fc9a3-7523-4262-b6ae-981a858b8493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:43.121350 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393fc9a3-7523-4262-b6ae-981a858b8493 admin admin] PATH_INFO: `/v3/projects/1e8f7a372f174c0698dbefd582aa8eec/users/92460ee9d7f64977bd0b93f2d54e4437/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:43.130679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:43.138422 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393fc9a3-7523-4262-b6ae-981a858b8493 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e8f7a372f174c0698dbefd582aa8eec, user_id=92460ee9d7f64977bd0b93f2d54e4437, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:43.142054 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393fc9a3-7523-4262-b6ae-981a858b8493 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:43.158198 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1261/2535] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:15:43 2022] PUT /identity/v3/projects/1e8f7a372f174c0698dbefd582aa8eec/users/92460ee9d7f64977bd0b93f2d54e4437/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:15:43.162793 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e703b-7782-44ae-8bde-1b1a63576d48 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:43.177754 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e703b-7782-44ae-8bde-1b1a63576d48 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:43.257327 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85668683-47bb-4a46-9bad-2c8885bad10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:43.257730 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85668683-47bb-4a46-9bad-2c8885bad10a admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:43.258036 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85668683-47bb-4a46-9bad-2c8885bad10a admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:43.258313 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85668683-47bb-4a46-9bad-2c8885bad10a admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:43.259049 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85668683-47bb-4a46-9bad-2c8885bad10a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:43.260723 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85668683-47bb-4a46-9bad-2c8885bad10a admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:43.272623 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1275/2536] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:15:43.277165 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393fc9a3-7523-4262-b6ae-981a858b8493 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:43.288983 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393fc9a3-7523-4262-b6ae-981a858b8493 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:43.295263 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb87d5-42ca-4b55-9ce9-52fb176dda40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:43.295950 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb87d5-42ca-4b55-9ce9-52fb176dda40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:43.296237 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb87d5-42ca-4b55-9ce9-52fb176dda40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:43.296492 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb87d5-42ca-4b55-9ce9-52fb176dda40 admin admin] PATH_INFO: `/v3/projects/1e8f7a372f174c0698dbefd582aa8eec/users/92460ee9d7f64977bd0b93f2d54e4437/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:43.298660 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bb87d5-42ca-4b55-9ce9-52fb176dda40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e8f7a372f174c0698dbefd582aa8eec, user_id=92460ee9d7f64977bd0b93f2d54e4437, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:43.301509 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bb87d5-42ca-4b55-9ce9-52fb176dda40 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:43.314375 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1262/2537] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:15:43 2022] PUT /identity/v3/projects/1e8f7a372f174c0698dbefd582aa8eec/users/92460ee9d7f64977bd0b93f2d54e4437/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:15:43.325743 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a78e8-db51-40a7-ad94-ddb353303767 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:43.326159 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a78e8-db51-40a7-ad94-ddb353303767 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:43.326508 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a78e8-db51-40a7-ad94-ddb353303767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:43.358678 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-ac7a78e8-db51-40a7-ad94-ddb353303767 None None] MFA Rules not processed for user `92460ee9d7f64977bd0b93f2d54e4437`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:15:43.360699 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-ac7a78e8-db51-40a7-ad94-ddb353303767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:43.375989 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:43.453948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1276/2538] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:15:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:15:43.467106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb87d5-42ca-4b55-9ce9-52fb176dda40 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:43.469876 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-e2bb87d5-42ca-4b55-9ce9-52fb176dda40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:43.545773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb87d5-42ca-4b55-9ce9-52fb176dda40 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:43.549505 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee084e-6c59-4f34-a6cc-d21b05a26d8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:43.550123 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee084e-6c59-4f34-a6cc-d21b05a26d8c service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:43.550421 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee084e-6c59-4f34-a6cc-d21b05a26d8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:43.550717 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee084e-6c59-4f34-a6cc-d21b05a26d8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:43.561285 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ee084e-6c59-4f34-a6cc-d21b05a26d8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:43.565555 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ee084e-6c59-4f34-a6cc-d21b05a26d8c service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:43.582899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1263/2539] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:15:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:15:43.622667 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a78e8-db51-40a7-ad94-ddb353303767 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:43.701679 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a78e8-db51-40a7-ad94-ddb353303767 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:43.707836 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f19f829-f6c4-46b2-999f-16f3e0756e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:43.708479 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19f829-f6c4-46b2-999f-16f3e0756e5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:43.708783 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19f829-f6c4-46b2-999f-16f3e0756e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:43.709053 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19f829-f6c4-46b2-999f-16f3e0756e5f admin admin] PATH_INFO: `/v3/users/64555def954448b2bd8fde44d11f7242` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:43.710223 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f19f829-f6c4-46b2-999f-16f3e0756e5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=64555def954448b2bd8fde44d11f7242)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:43.712176 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f19f829-f6c4-46b2-999f-16f3e0756e5f admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:43.760839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-1f19f829-f6c4-46b2-999f-16f3e0756e5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64555def954448b2bd8fde44d11f7242'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:43.769047 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-1f19f829-f6c4-46b2-999f-16f3e0756e5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64555def954448b2bd8fde44d11f7242'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:43.776079 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-1f19f829-f6c4-46b2-999f-16f3e0756e5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64555def954448b2bd8fde44d11f7242'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:43.789470 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-1f19f829-f6c4-46b2-999f-16f3e0756e5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64555def954448b2bd8fde44d11f7242'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:43.790155 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-1f19f829-f6c4-46b2-999f-16f3e0756e5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64555def954448b2bd8fde44d11f7242'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:43.796048 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1277/2540] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:15:43 2022] DELETE /identity/v3/users/64555def954448b2bd8fde44d11f7242 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:15:43.801667 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e266c-78ea-4926-b021-48c8c6b68891 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:43.802177 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e266c-78ea-4926-b021-48c8c6b68891 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:43.802279 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e266c-78ea-4926-b021-48c8c6b68891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:43.820884 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-e42e266c-78ea-4926-b021-48c8c6b68891 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:15:43.822065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-e42e266c-78ea-4926-b021-48c8c6b68891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:43.880523 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1264/2541] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:15:43.890544 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f19f829-f6c4-46b2-999f-16f3e0756e5f admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:43.892849 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-1f19f829-f6c4-46b2-999f-16f3e0756e5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:43.923558 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:43.951617 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f19f829-f6c4-46b2-999f-16f3e0756e5f admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:43.955978 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0706f3-c9cf-43ce-a4b1-dcedf3208f63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:43.956605 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0706f3-c9cf-43ce-a4b1-dcedf3208f63 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:43.956893 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0706f3-c9cf-43ce-a4b1-dcedf3208f63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:43.957163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0706f3-c9cf-43ce-a4b1-dcedf3208f63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:43.966357 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0706f3-c9cf-43ce-a4b1-dcedf3208f63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:43.967870 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0706f3-c9cf-43ce-a4b1-dcedf3208f63 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:43.977987 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1278/2542] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:15:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:15:44.257902 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e266c-78ea-4926-b021-48c8c6b68891 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:44.259391 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-e42e266c-78ea-4926-b021-48c8c6b68891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:44.272344 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e266c-78ea-4926-b021-48c8c6b68891 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:44.277714 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991fa02-3e8c-49e7-8b99-6dadb1370f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:44.278359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991fa02-3e8c-49e7-8b99-6dadb1370f2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:44.278670 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991fa02-3e8c-49e7-8b99-6dadb1370f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:44.279058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991fa02-3e8c-49e7-8b99-6dadb1370f2e admin admin] PATH_INFO: `/v3/projects/c9d01de6e09140ed9bc350fd0a4ea542` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:44.282701 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0991fa02-3e8c-49e7-8b99-6dadb1370f2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9d01de6e09140ed9bc350fd0a4ea542)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:44.298425 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0991fa02-3e8c-49e7-8b99-6dadb1370f2e admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:44.357634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-0991fa02-3e8c-49e7-8b99-6dadb1370f2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9d01de6e09140ed9bc350fd0a4ea542', 'request_id': 'req-0991fa02-3e8c-49e7-8b99-6dadb1370f2e'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:44.370941 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-0991fa02-3e8c-49e7-8b99-6dadb1370f2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9d01de6e09140ed9bc350fd0a4ea542', 'request_id': 'req-0991fa02-3e8c-49e7-8b99-6dadb1370f2e'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:44.377229 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-0991fa02-3e8c-49e7-8b99-6dadb1370f2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9d01de6e09140ed9bc350fd0a4ea542', 'request_id': 'req-0991fa02-3e8c-49e7-8b99-6dadb1370f2e'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:44.385417 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:44.389756 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-0991fa02-3e8c-49e7-8b99-6dadb1370f2e admin admin] The token cache is being invalidate because project c9d01de6e09140ed9bc350fd0a4ea542 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:15:44.390532 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-0991fa02-3e8c-49e7-8b99-6dadb1370f2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:44.391566 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-0991fa02-3e8c-49e7-8b99-6dadb1370f2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:44.393196 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1265/2543] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:15:44 2022] DELETE /identity/v3/projects/c9d01de6e09140ed9bc350fd0a4ea542 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:15:44.397561 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0706f3-c9cf-43ce-a4b1-dcedf3208f63 service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:44.408116 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0706f3-c9cf-43ce-a4b1-dcedf3208f63 service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:44.490609 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4844a34-b018-43af-a410-d2e453ada353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:44.491243 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4844a34-b018-43af-a410-d2e453ada353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:44.491529 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4844a34-b018-43af-a410-d2e453ada353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:44.491803 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4844a34-b018-43af-a410-d2e453ada353 admin admin] PATH_INFO: `/v3/users/92460ee9d7f64977bd0b93f2d54e4437` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:44.493023 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4844a34-b018-43af-a410-d2e453ada353 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92460ee9d7f64977bd0b93f2d54e4437)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:44.494951 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4844a34-b018-43af-a410-d2e453ada353 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:44.550968 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-c4844a34-b018-43af-a410-d2e453ada353 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92460ee9d7f64977bd0b93f2d54e4437'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:44.562875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-c4844a34-b018-43af-a410-d2e453ada353 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92460ee9d7f64977bd0b93f2d54e4437'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:44.575542 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-c4844a34-b018-43af-a410-d2e453ada353 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92460ee9d7f64977bd0b93f2d54e4437'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:44.594905 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-c4844a34-b018-43af-a410-d2e453ada353 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92460ee9d7f64977bd0b93f2d54e4437'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:44.596996 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-c4844a34-b018-43af-a410-d2e453ada353 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92460ee9d7f64977bd0b93f2d54e4437'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:44.605084 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1279/2544] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:15:44 2022] DELETE /identity/v3/users/92460ee9d7f64977bd0b93f2d54e4437 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:15:44.998652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991fa02-3e8c-49e7-8b99-6dadb1370f2e admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:45.001010 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-0991fa02-3e8c-49e7-8b99-6dadb1370f2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:45.021630 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991fa02-3e8c-49e7-8b99-6dadb1370f2e admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:45.095950 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198f6b2-7c1a-4d69-984d-97dc302f7a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:45.096629 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198f6b2-7c1a-4d69-984d-97dc302f7a7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:45.096919 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198f6b2-7c1a-4d69-984d-97dc302f7a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:45.097191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198f6b2-7c1a-4d69-984d-97dc302f7a7d admin admin] PATH_INFO: `/v3/projects/1e8f7a372f174c0698dbefd582aa8eec` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:45.098364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8198f6b2-7c1a-4d69-984d-97dc302f7a7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e8f7a372f174c0698dbefd582aa8eec)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:45.100287 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8198f6b2-7c1a-4d69-984d-97dc302f7a7d admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:45.160469 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8198f6b2-7c1a-4d69-984d-97dc302f7a7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e8f7a372f174c0698dbefd582aa8eec', 'request_id': 'req-8198f6b2-7c1a-4d69-984d-97dc302f7a7d'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:45.168126 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8198f6b2-7c1a-4d69-984d-97dc302f7a7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e8f7a372f174c0698dbefd582aa8eec', 'request_id': 'req-8198f6b2-7c1a-4d69-984d-97dc302f7a7d'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:45.175935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8198f6b2-7c1a-4d69-984d-97dc302f7a7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e8f7a372f174c0698dbefd582aa8eec', 'request_id': 'req-8198f6b2-7c1a-4d69-984d-97dc302f7a7d'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:45.192819 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8198f6b2-7c1a-4d69-984d-97dc302f7a7d admin admin] The token cache is being invalidate because project 1e8f7a372f174c0698dbefd582aa8eec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:15:45.193113 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8198f6b2-7c1a-4d69-984d-97dc302f7a7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:45.193773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8198f6b2-7c1a-4d69-984d-97dc302f7a7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:45.195304 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1266/2545] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:15:44 2022] DELETE /identity/v3/projects/1e8f7a372f174c0698dbefd582aa8eec => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:15:45.200228 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e55be-8322-4f92-9d35-4bffa3cc4eda None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:45.200516 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e55be-8322-4f92-9d35-4bffa3cc4eda None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:45.200831 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e55be-8322-4f92-9d35-4bffa3cc4eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:45.228392 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-6a6e55be-8322-4f92-9d35-4bffa3cc4eda None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:15:45.229678 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-6a6e55be-8322-4f92-9d35-4bffa3cc4eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:45.287781 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:45.306949 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1280/2546] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:15:45.315485 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5744fcd-2496-48da-a23c-8619c51a3709 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:45.315770 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5744fcd-2496-48da-a23c-8619c51a3709 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:45.316046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5744fcd-2496-48da-a23c-8619c51a3709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:45.340277 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-f5744fcd-2496-48da-a23c-8619c51a3709 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:15:45.341539 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-f5744fcd-2496-48da-a23c-8619c51a3709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:45.350999 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1267/2547] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:15:45.355454 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e55be-8322-4f92-9d35-4bffa3cc4eda None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:45.366860 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e55be-8322-4f92-9d35-4bffa3cc4eda None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:45.371508 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f19e1-6e71-41c8-a9b7-a67cfa5929f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:45.372112 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f19e1-6e71-41c8-a9b7-a67cfa5929f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:45.372389 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f19e1-6e71-41c8-a9b7-a67cfa5929f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:45.372690 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f19e1-6e71-41c8-a9b7-a67cfa5929f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:45.373378 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f19e1-6e71-41c8-a9b7-a67cfa5929f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:15:45.373680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f19e1-6e71-41c8-a9b7-a67cfa5929f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:45.375382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f19e1-6e71-41c8-a9b7-a67cfa5929f2 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:45.392513 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1281/2548] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:15:45.396678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5744fcd-2496-48da-a23c-8619c51a3709 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:45.410154 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5744fcd-2496-48da-a23c-8619c51a3709 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:45.417413 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f0ce2-2d35-472b-b54e-83c30f1fe233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:45.418161 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f0ce2-2d35-472b-b54e-83c30f1fe233 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:45.418526 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f0ce2-2d35-472b-b54e-83c30f1fe233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:45.418989 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f0ce2-2d35-472b-b54e-83c30f1fe233 admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:45.419738 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667f0ce2-2d35-472b-b54e-83c30f1fe233 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:45.423410 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667f0ce2-2d35-472b-b54e-83c30f1fe233 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:45.456613 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1268/2549] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:15:45 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:15:45.460875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f19e1-6e71-41c8-a9b7-a67cfa5929f2 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:45.473252 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f19e1-6e71-41c8-a9b7-a67cfa5929f2 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:45.486144 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f0ce2-2d35-472b-b54e-83c30f1fe233 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:45.488603 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-667f0ce2-2d35-472b-b54e-83c30f1fe233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:45.563349 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b332fd-7a82-45bf-a523-5534f3807003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:45.564170 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b332fd-7a82-45bf-a523-5534f3807003 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:45.564517 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b332fd-7a82-45bf-a523-5534f3807003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:45.564889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b332fd-7a82-45bf-a523-5534f3807003 admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:45.564993 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f0ce2-2d35-472b-b54e-83c30f1fe233 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:45.565639 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b332fd-7a82-45bf-a523-5534f3807003 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:45.567588 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b332fd-7a82-45bf-a523-5534f3807003 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:45.569030 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d322b-02b3-49e2-9e79-f2159c802b7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:45.569669 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d322b-02b3-49e2-9e79-f2159c802b7b service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:45.569954 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d322b-02b3-49e2-9e79-f2159c802b7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:45.570408 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d322b-02b3-49e2-9e79-f2159c802b7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:45.572101 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-fe6d322b-02b3-49e2-9e79-f2159c802b7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:45.588729 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:45.595937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1282/2550] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:15:45 2022] POST /identity/v3/users => generated 292 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:15:45.600869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b332fd-7a82-45bf-a523-5534f3807003 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:45.612355 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b332fd-7a82-45bf-a523-5534f3807003 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:45.617074 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cffa786-1984-4ca2-8d07-3d992bf8e0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:45.617920 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffa786-1984-4ca2-8d07-3d992bf8e0ce admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:45.618203 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffa786-1984-4ca2-8d07-3d992bf8e0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:45.618475 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffa786-1984-4ca2-8d07-3d992bf8e0ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:45.619166 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cffa786-1984-4ca2-8d07-3d992bf8e0ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:45.620903 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cffa786-1984-4ca2-8d07-3d992bf8e0ce admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:45.631893 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1283/2551] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:15:45.635880 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cffa786-1984-4ca2-8d07-3d992bf8e0ce admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:45.646903 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cffa786-1984-4ca2-8d07-3d992bf8e0ce admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:45.651936 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262227d-4618-43b1-883b-31a58c5e33df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:45.652564 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262227d-4618-43b1-883b-31a58c5e33df admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:45.652874 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262227d-4618-43b1-883b-31a58c5e33df admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:45.653145 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262227d-4618-43b1-883b-31a58c5e33df admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:45.653991 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9262227d-4618-43b1-883b-31a58c5e33df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:45.655620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9262227d-4618-43b1-883b-31a58c5e33df admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:45.666864 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1284/2552] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:15:45.671574 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262227d-4618-43b1-883b-31a58c5e33df admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:45.673466 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6d322b-02b3-49e2-9e79-f2159c802b7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:45.675121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6d322b-02b3-49e2-9e79-f2159c802b7b service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:45.683801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262227d-4618-43b1-883b-31a58c5e33df admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:45.689215 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7886c-fbd2-4476-ad1b-f6a800c37f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:45.689887 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7886c-fbd2-4476-ad1b-f6a800c37f34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:45.690164 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7886c-fbd2-4476-ad1b-f6a800c37f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:45.690439 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7886c-fbd2-4476-ad1b-f6a800c37f34 admin admin] PATH_INFO: `/v3/projects/8690e63c3a7842c098afb1663fc04ffc/users/832eaa236a294b05acbd02d15caf033f/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:45.695581 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1269/2553] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:15:45 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:15:45.707025 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7886c-fbd2-4476-ad1b-f6a800c37f34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8690e63c3a7842c098afb1663fc04ffc, user_id=832eaa236a294b05acbd02d15caf033f, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:45.709864 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7886c-fbd2-4476-ad1b-f6a800c37f34 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:45.729353 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1285/2554] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:15:45 2022] PUT /identity/v3/projects/8690e63c3a7842c098afb1663fc04ffc/users/832eaa236a294b05acbd02d15caf033f/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:15:45.743405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50071743-c91d-469e-b316-5e117417784c None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:45.744033 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50071743-c91d-469e-b316-5e117417784c None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:45.744637 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50071743-c91d-469e-b316-5e117417784c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:45.787520 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-50071743-c91d-469e-b316-5e117417784c None None] MFA Rules not processed for user `832eaa236a294b05acbd02d15caf033f`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:15:45.789701 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-50071743-c91d-469e-b316-5e117417784c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:45.889229 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1270/2555] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:15:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:15:45.893941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7886c-fbd2-4476-ad1b-f6a800c37f34 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:45.994221 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7886c-fbd2-4476-ad1b-f6a800c37f34 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:46.000729 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec98721e-38a9-4ee1-9da9-7ab1f2175d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:46.001421 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98721e-38a9-4ee1-9da9-7ab1f2175d8d admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:46.001708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98721e-38a9-4ee1-9da9-7ab1f2175d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:46.002520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98721e-38a9-4ee1-9da9-7ab1f2175d8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:46.002702 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec98721e-38a9-4ee1-9da9-7ab1f2175d8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:46.004657 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec98721e-38a9-4ee1-9da9-7ab1f2175d8d admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:46.056454 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1286/2556] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:15:45 2022] POST /identity/v3/projects => generated 383 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:15:46.060420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50071743-c91d-469e-b316-5e117417784c None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:46.074818 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50071743-c91d-469e-b316-5e117417784c None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:46.168102 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97624cd7-1362-4efb-b2ac-a205e3ceb482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:46.169217 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97624cd7-1362-4efb-b2ac-a205e3ceb482 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:46.169720 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97624cd7-1362-4efb-b2ac-a205e3ceb482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:46.170204 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97624cd7-1362-4efb-b2ac-a205e3ceb482 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:46.171268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97624cd7-1362-4efb-b2ac-a205e3ceb482 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:46.178732 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97624cd7-1362-4efb-b2ac-a205e3ceb482 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:46.202133 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1271/2557] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:15:46 2022] POST /identity/v3/users => generated 298 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:15:46.208767 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec98721e-38a9-4ee1-9da9-7ab1f2175d8d admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:46.216568 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:46.221994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec98721e-38a9-4ee1-9da9-7ab1f2175d8d admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:46.232001 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de8822-049b-4679-af5c-e0e90487289a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:46.232742 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de8822-049b-4679-af5c-e0e90487289a admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:46.233032 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de8822-049b-4679-af5c-e0e90487289a admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:46.233306 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de8822-049b-4679-af5c-e0e90487289a admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:46.234007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de8822-049b-4679-af5c-e0e90487289a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:46.236447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de8822-049b-4679-af5c-e0e90487289a admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:46.247681 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1287/2558] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:15:46.253011 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97624cd7-1362-4efb-b2ac-a205e3ceb482 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:46.263076 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97624cd7-1362-4efb-b2ac-a205e3ceb482 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:46.268672 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa5e3f-b2eb-4fbb-8258-4774def91844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:46.269612 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa5e3f-b2eb-4fbb-8258-4774def91844 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:46.269888 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa5e3f-b2eb-4fbb-8258-4774def91844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:46.270162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa5e3f-b2eb-4fbb-8258-4774def91844 admin admin] PATH_INFO: `/v3/projects/3431c2d5206d495480f80ea4b0316b8f/users/f5b7c860517a4dd8be087093dae7bfaf/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:46.272531 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:46.288448 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fa5e3f-b2eb-4fbb-8258-4774def91844 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3431c2d5206d495480f80ea4b0316b8f, user_id=f5b7c860517a4dd8be087093dae7bfaf, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:46.291238 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fa5e3f-b2eb-4fbb-8258-4774def91844 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:46.305864 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1272/2559] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:15:46 2022] PUT /identity/v3/projects/3431c2d5206d495480f80ea4b0316b8f/users/f5b7c860517a4dd8be087093dae7bfaf/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:15:46.310296 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de8822-049b-4679-af5c-e0e90487289a admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:46.327627 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de8822-049b-4679-af5c-e0e90487289a admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:46.392530 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b87f723-34b1-4159-9418-e1ba7d25ae7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:46.393176 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87f723-34b1-4159-9418-e1ba7d25ae7f admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:46.393459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87f723-34b1-4159-9418-e1ba7d25ae7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:46.393999 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87f723-34b1-4159-9418-e1ba7d25ae7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:46.394701 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b87f723-34b1-4159-9418-e1ba7d25ae7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:46.396306 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b87f723-34b1-4159-9418-e1ba7d25ae7f admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:46.407593 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1288/2560] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:15:46.412012 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa5e3f-b2eb-4fbb-8258-4774def91844 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:46.423961 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa5e3f-b2eb-4fbb-8258-4774def91844 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:46.428700 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64781966-9736-42a9-9303-fb08245a0377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:46.429364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64781966-9736-42a9-9303-fb08245a0377 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:46.429791 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64781966-9736-42a9-9303-fb08245a0377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:46.430108 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64781966-9736-42a9-9303-fb08245a0377 admin admin] PATH_INFO: `/v3/projects/3431c2d5206d495480f80ea4b0316b8f/users/f5b7c860517a4dd8be087093dae7bfaf/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:46.432219 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64781966-9736-42a9-9303-fb08245a0377 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3431c2d5206d495480f80ea4b0316b8f, user_id=f5b7c860517a4dd8be087093dae7bfaf, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:46.435119 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64781966-9736-42a9-9303-fb08245a0377 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:46.446652 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1273/2561] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:15:46 2022] PUT /identity/v3/projects/3431c2d5206d495480f80ea4b0316b8f/users/f5b7c860517a4dd8be087093dae7bfaf/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:15:46.457590 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ddda3f-1ffd-49af-a976-717234ba8c04 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:46.457932 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ddda3f-1ffd-49af-a976-717234ba8c04 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:46.458319 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ddda3f-1ffd-49af-a976-717234ba8c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:46.489661 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-d1ddda3f-1ffd-49af-a976-717234ba8c04 None None] MFA Rules not processed for user `f5b7c860517a4dd8be087093dae7bfaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:15:46.490881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-d1ddda3f-1ffd-49af-a976-717234ba8c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:46.567568 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1289/2562] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:15:46 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:15:46.581742 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64781966-9736-42a9-9303-fb08245a0377 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:46.583463 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-64781966-9736-42a9-9303-fb08245a0377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:46.646104 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64781966-9736-42a9-9303-fb08245a0377 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:46.649908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb298a6c-ac15-4689-8c07-6f20df3f1975 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:46.650536 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb298a6c-ac15-4689-8c07-6f20df3f1975 service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:46.650816 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb298a6c-ac15-4689-8c07-6f20df3f1975 service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:46.651090 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb298a6c-ac15-4689-8c07-6f20df3f1975 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:46.661800 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb298a6c-ac15-4689-8c07-6f20df3f1975 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:46.663235 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb298a6c-ac15-4689-8c07-6f20df3f1975 service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:46.674496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1274/2563] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:15:46 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:15:46.821907 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ddda3f-1ffd-49af-a976-717234ba8c04 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:46.839183 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ddda3f-1ffd-49af-a976-717234ba8c04 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:46.845106 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a4ed3-53f9-4f55-ac87-60f05f11618c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:46.845974 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a4ed3-53f9-4f55-ac87-60f05f11618c service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:46.846385 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a4ed3-53f9-4f55-ac87-60f05f11618c service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:46.846840 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a4ed3-53f9-4f55-ac87-60f05f11618c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:46.931728 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a4ed3-53f9-4f55-ac87-60f05f11618c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:46.934466 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a4ed3-53f9-4f55-ac87-60f05f11618c service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:46.949460 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1290/2564] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:15:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:15:46.996337 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d8969-dd9b-4dac-89df-ee5e2b77be59 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:46.996848 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d8969-dd9b-4dac-89df-ee5e2b77be59 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:46.997330 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d8969-dd9b-4dac-89df-ee5e2b77be59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:47.021456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-583d8969-dd9b-4dac-89df-ee5e2b77be59 None None] MFA Rules not processed for user `b5d959f3af3745ed8e0bc4a67faf34f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:15:47.023422 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-583d8969-dd9b-4dac-89df-ee5e2b77be59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:47.098308 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1275/2565] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:15:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:15:47.648384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a4ed3-53f9-4f55-ac87-60f05f11618c service cinder] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:47.650682 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-957a4ed3-53f9-4f55-ac87-60f05f11618c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:47.675196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:47.729684 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a4ed3-53f9-4f55-ac87-60f05f11618c service cinder] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:47.733759 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f51d3-f889-4693-8b41-4f02495f8199 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:47.734441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f51d3-f889-4693-8b41-4f02495f8199 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:47.735397 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f51d3-f889-4693-8b41-4f02495f8199 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:47.735823 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f51d3-f889-4693-8b41-4f02495f8199 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:47.737883 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-355f51d3-f889-4693-8b41-4f02495f8199 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:47.749867 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f51d3-f889-4693-8b41-4f02495f8199 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:47.751678 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f51d3-f889-4693-8b41-4f02495f8199 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:47.762196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1291/2566] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:15:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:15:49.848400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d8969-dd9b-4dac-89df-ee5e2b77be59 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:50.044047 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d8969-dd9b-4dac-89df-ee5e2b77be59 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:50.050219 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df26c8-f569-4a4a-b966-9709a98cefd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:50.050876 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df26c8-f569-4a4a-b966-9709a98cefd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:50.051147 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df26c8-f569-4a4a-b966-9709a98cefd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:50.051588 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df26c8-f569-4a4a-b966-9709a98cefd8 admin admin] PATH_INFO: `/v3/users/832eaa236a294b05acbd02d15caf033f` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:50.053038 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df26c8-f569-4a4a-b966-9709a98cefd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=832eaa236a294b05acbd02d15caf033f)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:50.055256 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df26c8-f569-4a4a-b966-9709a98cefd8 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:50.092475 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:50.166148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-23df26c8-f569-4a4a-b966-9709a98cefd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '832eaa236a294b05acbd02d15caf033f'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:50.181859 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-23df26c8-f569-4a4a-b966-9709a98cefd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '832eaa236a294b05acbd02d15caf033f'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:50.225158 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-23df26c8-f569-4a4a-b966-9709a98cefd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '832eaa236a294b05acbd02d15caf033f'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:50.226424 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-23df26c8-f569-4a4a-b966-9709a98cefd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '832eaa236a294b05acbd02d15caf033f'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:50.244087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-23df26c8-f569-4a4a-b966-9709a98cefd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '832eaa236a294b05acbd02d15caf033f'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:50.267789 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1276/2567] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:15:49 2022] DELETE /identity/v3/users/832eaa236a294b05acbd02d15caf033f => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:15:50.269129 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d0f1c-875c-4f03-8d3d-d2204aeda201 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:50.269976 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d0f1c-875c-4f03-8d3d-d2204aeda201 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:50.270544 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d0f1c-875c-4f03-8d3d-d2204aeda201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:50.291690 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-624d0f1c-875c-4f03-8d3d-d2204aeda201 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:15:50.293005 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-624d0f1c-875c-4f03-8d3d-d2204aeda201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:50.398301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1292/2568] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:15:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:15:50.410670 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df26c8-f569-4a4a-b966-9709a98cefd8 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:50.412406 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-23df26c8-f569-4a4a-b966-9709a98cefd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:50.546257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df26c8-f569-4a4a-b966-9709a98cefd8 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:50.551843 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea4ebe-1247-4df8-8bcf-1fd9f4828eff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:50.553281 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea4ebe-1247-4df8-8bcf-1fd9f4828eff service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:50.553868 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea4ebe-1247-4df8-8bcf-1fd9f4828eff service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:50.554326 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea4ebe-1247-4df8-8bcf-1fd9f4828eff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:50.569912 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ea4ebe-1247-4df8-8bcf-1fd9f4828eff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:50.572715 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ea4ebe-1247-4df8-8bcf-1fd9f4828eff service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:50.600719 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1277/2569] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:15:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:15:51.163608 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d0f1c-875c-4f03-8d3d-d2204aeda201 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:51.165191 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-624d0f1c-875c-4f03-8d3d-d2204aeda201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:51.181914 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d0f1c-875c-4f03-8d3d-d2204aeda201 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:51.205124 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b02e7c6-07dd-4f08-8af6-4a482dd7e45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:51.205811 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02e7c6-07dd-4f08-8af6-4a482dd7e45f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:51.206119 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02e7c6-07dd-4f08-8af6-4a482dd7e45f admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:51.206301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02e7c6-07dd-4f08-8af6-4a482dd7e45f admin admin] PATH_INFO: `/v3/projects/8690e63c3a7842c098afb1663fc04ffc` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:51.208433 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b02e7c6-07dd-4f08-8af6-4a482dd7e45f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8690e63c3a7842c098afb1663fc04ffc)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:51.210859 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b02e7c6-07dd-4f08-8af6-4a482dd7e45f admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:51.292837 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-7b02e7c6-07dd-4f08-8af6-4a482dd7e45f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8690e63c3a7842c098afb1663fc04ffc', 'request_id': 'req-7b02e7c6-07dd-4f08-8af6-4a482dd7e45f'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:51.313640 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-7b02e7c6-07dd-4f08-8af6-4a482dd7e45f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8690e63c3a7842c098afb1663fc04ffc', 'request_id': 'req-7b02e7c6-07dd-4f08-8af6-4a482dd7e45f'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:51.327134 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-7b02e7c6-07dd-4f08-8af6-4a482dd7e45f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8690e63c3a7842c098afb1663fc04ffc', 'request_id': 'req-7b02e7c6-07dd-4f08-8af6-4a482dd7e45f'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:51.351415 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-7b02e7c6-07dd-4f08-8af6-4a482dd7e45f admin admin] The token cache is being invalidate because project 8690e63c3a7842c098afb1663fc04ffc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:15:51.351900 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-7b02e7c6-07dd-4f08-8af6-4a482dd7e45f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:51.353096 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-7b02e7c6-07dd-4f08-8af6-4a482dd7e45f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:51.356884 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1293/2570] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:15:51 2022] DELETE /identity/v3/projects/8690e63c3a7842c098afb1663fc04ffc => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:15:51.361964 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea4ebe-1247-4df8-8bcf-1fd9f4828eff service neutron] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:51.397100 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea4ebe-1247-4df8-8bcf-1fd9f4828eff service neutron] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:51.544511 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2618d5-4df3-4e79-9f1a-c7a271203ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:51.545896 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2618d5-4df3-4e79-9f1a-c7a271203ff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:51.546482 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2618d5-4df3-4e79-9f1a-c7a271203ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:51.547048 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2618d5-4df3-4e79-9f1a-c7a271203ff3 admin admin] PATH_INFO: `/v3/users/f5b7c860517a4dd8be087093dae7bfaf` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:51.549603 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2618d5-4df3-4e79-9f1a-c7a271203ff3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5b7c860517a4dd8be087093dae7bfaf)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:51.552109 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2618d5-4df3-4e79-9f1a-c7a271203ff3 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:51.668330 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:51.692526 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-6e2618d5-4df3-4e79-9f1a-c7a271203ff3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5b7c860517a4dd8be087093dae7bfaf'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:51.710946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-6e2618d5-4df3-4e79-9f1a-c7a271203ff3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5b7c860517a4dd8be087093dae7bfaf'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:51.750882 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-6e2618d5-4df3-4e79-9f1a-c7a271203ff3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5b7c860517a4dd8be087093dae7bfaf'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:51.752442 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-6e2618d5-4df3-4e79-9f1a-c7a271203ff3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5b7c860517a4dd8be087093dae7bfaf'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:51.778256 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-6e2618d5-4df3-4e79-9f1a-c7a271203ff3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5b7c860517a4dd8be087093dae7bfaf'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:51.807466 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1278/2571] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:15:51 2022] DELETE /identity/v3/users/f5b7c860517a4dd8be087093dae7bfaf => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:15:52.547946 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b02e7c6-07dd-4f08-8af6-4a482dd7e45f admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:52.550426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-7b02e7c6-07dd-4f08-8af6-4a482dd7e45f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:52.578513 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b02e7c6-07dd-4f08-8af6-4a482dd7e45f admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:52.696626 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:52.751583 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ba45f6-85e5-428a-9027-bf46b2980004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:52.753250 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba45f6-85e5-428a-9027-bf46b2980004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:52.753897 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba45f6-85e5-428a-9027-bf46b2980004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:52.754488 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba45f6-85e5-428a-9027-bf46b2980004 admin admin] PATH_INFO: `/v3/projects/3431c2d5206d495480f80ea4b0316b8f` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:52.757928 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ba45f6-85e5-428a-9027-bf46b2980004 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3431c2d5206d495480f80ea4b0316b8f)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:52.763041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ba45f6-85e5-428a-9027-bf46b2980004 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:52.897932 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-d9ba45f6-85e5-428a-9027-bf46b2980004 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3431c2d5206d495480f80ea4b0316b8f', 'request_id': 'req-d9ba45f6-85e5-428a-9027-bf46b2980004'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:52.907526 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-d9ba45f6-85e5-428a-9027-bf46b2980004 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3431c2d5206d495480f80ea4b0316b8f', 'request_id': 'req-d9ba45f6-85e5-428a-9027-bf46b2980004'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:52.924632 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-d9ba45f6-85e5-428a-9027-bf46b2980004 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3431c2d5206d495480f80ea4b0316b8f', 'request_id': 'req-d9ba45f6-85e5-428a-9027-bf46b2980004'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:52.938755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-d9ba45f6-85e5-428a-9027-bf46b2980004 admin admin] The token cache is being invalidate because project 3431c2d5206d495480f80ea4b0316b8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:15:52.939421 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-d9ba45f6-85e5-428a-9027-bf46b2980004 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:52.940795 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-d9ba45f6-85e5-428a-9027-bf46b2980004 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:52.948719 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1294/2572] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:15:52 2022] DELETE /identity/v3/projects/3431c2d5206d495480f80ea4b0316b8f => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:15:52.950142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e5e56-da0e-47d3-a33d-00b8ac944802 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:52.950673 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e5e56-da0e-47d3-a33d-00b8ac944802 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:52.950673 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e5e56-da0e-47d3-a33d-00b8ac944802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:52.978934 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-1e3e5e56-da0e-47d3-a33d-00b8ac944802 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:15:52.980875 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-1e3e5e56-da0e-47d3-a33d-00b8ac944802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:53.074372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1279/2573] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:15:53.084670 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691749a3-4380-4dff-b136-7940a16288bf None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:53.084961 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691749a3-4380-4dff-b136-7940a16288bf None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:53.085231 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691749a3-4380-4dff-b136-7940a16288bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:53.114417 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-691749a3-4380-4dff-b136-7940a16288bf None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:15:53.116904 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-691749a3-4380-4dff-b136-7940a16288bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:53.126494 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1295/2574] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:15:53.131564 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e5e56-da0e-47d3-a33d-00b8ac944802 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:53.148812 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e5e56-da0e-47d3-a33d-00b8ac944802 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:53.156236 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291a9b27-e850-48e4-8850-ccf6a7896354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:53.158085 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291a9b27-e850-48e4-8850-ccf6a7896354 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:53.158582 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291a9b27-e850-48e4-8850-ccf6a7896354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:53.159070 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291a9b27-e850-48e4-8850-ccf6a7896354 admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:53.160499 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291a9b27-e850-48e4-8850-ccf6a7896354 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:15:53.161607 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291a9b27-e850-48e4-8850-ccf6a7896354 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:53.163837 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291a9b27-e850-48e4-8850-ccf6a7896354 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:53.188825 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1280/2575] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:15:53.191031 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691749a3-4380-4dff-b136-7940a16288bf None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:53.201412 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691749a3-4380-4dff-b136-7940a16288bf None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:53.206895 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374bb386-74fe-4dac-af1e-8ad7f145b443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:53.207669 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374bb386-74fe-4dac-af1e-8ad7f145b443 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:53.208009 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374bb386-74fe-4dac-af1e-8ad7f145b443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:53.208264 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374bb386-74fe-4dac-af1e-8ad7f145b443 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:53.208919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374bb386-74fe-4dac-af1e-8ad7f145b443 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:53.210832 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374bb386-74fe-4dac-af1e-8ad7f145b443 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:53.257318 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1296/2576] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:15:53 2022] POST /identity/v3/projects => generated 387 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:15:53.258774 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291a9b27-e850-48e4-8850-ccf6a7896354 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:53.280335 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291a9b27-e850-48e4-8850-ccf6a7896354 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:53.357538 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e519c9-5b34-40b6-80b0-3dfd7cc165c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:53.358497 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e519c9-5b34-40b6-80b0-3dfd7cc165c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:53.358938 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e519c9-5b34-40b6-80b0-3dfd7cc165c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:53.359501 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e519c9-5b34-40b6-80b0-3dfd7cc165c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:53.361365 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e519c9-5b34-40b6-80b0-3dfd7cc165c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:53.365219 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e519c9-5b34-40b6-80b0-3dfd7cc165c2 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:53.399073 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1281/2577] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:15:53 2022] POST /identity/v3/users => generated 294 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:15:53.405487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374bb386-74fe-4dac-af1e-8ad7f145b443 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:53.420316 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374bb386-74fe-4dac-af1e-8ad7f145b443 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:53.427854 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a2ca8-62d4-4d1b-a76e-b755c6e32339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:53.428966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a2ca8-62d4-4d1b-a76e-b755c6e32339 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:53.429425 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a2ca8-62d4-4d1b-a76e-b755c6e32339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:53.429947 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a2ca8-62d4-4d1b-a76e-b755c6e32339 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:53.431269 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a2ca8-62d4-4d1b-a76e-b755c6e32339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:53.434520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a2ca8-62d4-4d1b-a76e-b755c6e32339 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:53.453329 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1297/2578] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:15:53.458146 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e519c9-5b34-40b6-80b0-3dfd7cc165c2 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:53.471287 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e519c9-5b34-40b6-80b0-3dfd7cc165c2 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:53.476903 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c3573-14e1-4857-bd82-2b83d7081867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:53.477571 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c3573-14e1-4857-bd82-2b83d7081867 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:53.477854 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c3573-14e1-4857-bd82-2b83d7081867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:53.478128 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c3573-14e1-4857-bd82-2b83d7081867 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:53.478866 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c3573-14e1-4857-bd82-2b83d7081867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:53.480787 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c3573-14e1-4857-bd82-2b83d7081867 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:53.506170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1282/2579] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:15:53.512155 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a2ca8-62d4-4d1b-a76e-b755c6e32339 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:53.523838 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a2ca8-62d4-4d1b-a76e-b755c6e32339 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:53.528977 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c590a9-3410-4f27-b377-de20be53ca48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:53.529791 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c590a9-3410-4f27-b377-de20be53ca48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:53.530069 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c590a9-3410-4f27-b377-de20be53ca48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:53.530346 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c590a9-3410-4f27-b377-de20be53ca48 admin admin] PATH_INFO: `/v3/projects/e0b1916130814fa4a2462683e362cac4/users/0c8cdbb5b60b478a9155cbad581e6faa/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:53.552650 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c590a9-3410-4f27-b377-de20be53ca48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0b1916130814fa4a2462683e362cac4, user_id=0c8cdbb5b60b478a9155cbad581e6faa, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:53.555869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c590a9-3410-4f27-b377-de20be53ca48 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:53.577854 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1298/2580] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:15:53 2022] PUT /identity/v3/projects/e0b1916130814fa4a2462683e362cac4/users/0c8cdbb5b60b478a9155cbad581e6faa/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:15:53.596314 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca69731-d706-4bba-a006-fae5e3b2cbab None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:53.596727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca69731-d706-4bba-a006-fae5e3b2cbab None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:53.597135 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca69731-d706-4bba-a006-fae5e3b2cbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:53.632783 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-7ca69731-d706-4bba-a006-fae5e3b2cbab None None] MFA Rules not processed for user `0c8cdbb5b60b478a9155cbad581e6faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:15:53.634577 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-7ca69731-d706-4bba-a006-fae5e3b2cbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:53.652402 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:53.743180 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1283/2581] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:15:53 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:15:53.749862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c590a9-3410-4f27-b377-de20be53ca48 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:53.863625 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c590a9-3410-4f27-b377-de20be53ca48 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:53.871205 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba910ce-cbf0-4f92-87ad-a953fdf56030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:53.872050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba910ce-cbf0-4f92-87ad-a953fdf56030 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:53.872523 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba910ce-cbf0-4f92-87ad-a953fdf56030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:53.873075 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba910ce-cbf0-4f92-87ad-a953fdf56030 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:53.874347 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba910ce-cbf0-4f92-87ad-a953fdf56030 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:53.877860 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba910ce-cbf0-4f92-87ad-a953fdf56030 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:53.894889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:53.918249 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1299/2582] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:15:53 2022] POST /identity/v3/projects => generated 385 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:15:53.924983 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca69731-d706-4bba-a006-fae5e3b2cbab None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:53.939378 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca69731-d706-4bba-a006-fae5e3b2cbab None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:54.026035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a64d16-20aa-4f64-84b6-5c599ef278a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:54.027076 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a64d16-20aa-4f64-84b6-5c599ef278a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:54.027533 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a64d16-20aa-4f64-84b6-5c599ef278a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:54.027906 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a64d16-20aa-4f64-84b6-5c599ef278a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:54.029182 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a64d16-20aa-4f64-84b6-5c599ef278a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:54.031969 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a64d16-20aa-4f64-84b6-5c599ef278a4 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:54.079290 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1284/2583] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:15:53 2022] POST /identity/v3/users => generated 299 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:15:54.085456 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba910ce-cbf0-4f92-87ad-a953fdf56030 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:54.104473 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba910ce-cbf0-4f92-87ad-a953fdf56030 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:54.119473 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283cc80-b32a-46c4-af21-fddb899da5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:54.120239 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283cc80-b32a-46c4-af21-fddb899da5f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:54.120592 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283cc80-b32a-46c4-af21-fddb899da5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:54.120953 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283cc80-b32a-46c4-af21-fddb899da5f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:54.121719 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1283cc80-b32a-46c4-af21-fddb899da5f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:54.123404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1283cc80-b32a-46c4-af21-fddb899da5f2 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:54.135448 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1300/2584] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:15:54.140158 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a64d16-20aa-4f64-84b6-5c599ef278a4 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:54.164627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a64d16-20aa-4f64-84b6-5c599ef278a4 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:54.173516 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad47e8e6-6c1a-44b8-b3ac-a3895a25a7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:54.174989 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad47e8e6-6c1a-44b8-b3ac-a3895a25a7a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:54.175558 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad47e8e6-6c1a-44b8-b3ac-a3895a25a7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:54.176112 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad47e8e6-6c1a-44b8-b3ac-a3895a25a7a7 admin admin] PATH_INFO: `/v3/projects/f70812d542a748f982a7cd3e2c203ebe/users/7f955e9ee1704b39b570e83d30450267/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:54.204804 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad47e8e6-6c1a-44b8-b3ac-a3895a25a7a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f70812d542a748f982a7cd3e2c203ebe, user_id=7f955e9ee1704b39b570e83d30450267, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:54.207633 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad47e8e6-6c1a-44b8-b3ac-a3895a25a7a7 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:54.235901 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1285/2585] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:15:54 2022] PUT /identity/v3/projects/f70812d542a748f982a7cd3e2c203ebe/users/7f955e9ee1704b39b570e83d30450267/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:15:54.242150 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283cc80-b32a-46c4-af21-fddb899da5f2 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:54.267274 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283cc80-b32a-46c4-af21-fddb899da5f2 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:54.340501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add348ff-2bbc-4401-a231-3c7ec66188ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:54.341415 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add348ff-2bbc-4401-a231-3c7ec66188ef admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:54.341790 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add348ff-2bbc-4401-a231-3c7ec66188ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:54.342190 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add348ff-2bbc-4401-a231-3c7ec66188ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:54.342986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add348ff-2bbc-4401-a231-3c7ec66188ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:54.344960 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add348ff-2bbc-4401-a231-3c7ec66188ef admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:54.356401 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1301/2586] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:15:54.365745 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad47e8e6-6c1a-44b8-b3ac-a3895a25a7a7 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:54.377945 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad47e8e6-6c1a-44b8-b3ac-a3895a25a7a7 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:54.383304 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115454b-76cd-4241-a2ed-65549e7d2169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:54.384012 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115454b-76cd-4241-a2ed-65549e7d2169 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:54.384341 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115454b-76cd-4241-a2ed-65549e7d2169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:54.384693 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115454b-76cd-4241-a2ed-65549e7d2169 admin admin] PATH_INFO: `/v3/projects/f70812d542a748f982a7cd3e2c203ebe/users/7f955e9ee1704b39b570e83d30450267/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:54.386772 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115454b-76cd-4241-a2ed-65549e7d2169 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f70812d542a748f982a7cd3e2c203ebe, user_id=7f955e9ee1704b39b570e83d30450267, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:54.389443 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115454b-76cd-4241-a2ed-65549e7d2169 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:54.402864 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1286/2587] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:15:54 2022] PUT /identity/v3/projects/f70812d542a748f982a7cd3e2c203ebe/users/7f955e9ee1704b39b570e83d30450267/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:15:54.414667 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d3388-d1d8-4d23-9961-cf3a31df39af None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:54.415017 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d3388-d1d8-4d23-9961-cf3a31df39af None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:54.415337 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d3388-d1d8-4d23-9961-cf3a31df39af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:54.446007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-df4d3388-d1d8-4d23-9961-cf3a31df39af None None] MFA Rules not processed for user `7f955e9ee1704b39b570e83d30450267`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:15:54.447280 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-df4d3388-d1d8-4d23-9961-cf3a31df39af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:54.541665 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1302/2588] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:15:54 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:15:54.555556 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115454b-76cd-4241-a2ed-65549e7d2169 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:54.558147 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-7115454b-76cd-4241-a2ed-65549e7d2169 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:54.787213 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115454b-76cd-4241-a2ed-65549e7d2169 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:54.791840 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba094c0-e679-4a34-9ba5-70c85d6ddbf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:54.792354 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba094c0-e679-4a34-9ba5-70c85d6ddbf6 service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:54.792671 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba094c0-e679-4a34-9ba5-70c85d6ddbf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:54.792944 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba094c0-e679-4a34-9ba5-70c85d6ddbf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:54.806844 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba094c0-e679-4a34-9ba5-70c85d6ddbf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:54.809672 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba094c0-e679-4a34-9ba5-70c85d6ddbf6 service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:54.827827 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1287/2589] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:15:54 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:15:55.514791 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d3388-d1d8-4d23-9961-cf3a31df39af None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:55.516865 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-df4d3388-d1d8-4d23-9961-cf3a31df39af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:55.634252 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d3388-d1d8-4d23-9961-cf3a31df39af None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:55.641976 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbceeab-8c71-43da-8918-746fff9e5b9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:55.642802 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbceeab-8c71-43da-8918-746fff9e5b9a service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:55.644361 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbceeab-8c71-43da-8918-746fff9e5b9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:55.644856 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbceeab-8c71-43da-8918-746fff9e5b9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:55.649253 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-2dbceeab-8c71-43da-8918-746fff9e5b9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:55.671105 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:55.751644 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbceeab-8c71-43da-8918-746fff9e5b9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:55.753467 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbceeab-8c71-43da-8918-746fff9e5b9a service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:55.775548 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1303/2590] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:15:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:15:55.893525 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba094c0-e679-4a34-9ba5-70c85d6ddbf6 service cinder] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:55.907237 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba094c0-e679-4a34-9ba5-70c85d6ddbf6 service cinder] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:55.911958 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea46a1f5-00aa-4c04-b240-a0d73f154c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:55.912628 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46a1f5-00aa-4c04-b240-a0d73f154c05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:55.912908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46a1f5-00aa-4c04-b240-a0d73f154c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:55.913387 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46a1f5-00aa-4c04-b240-a0d73f154c05 admin admin] PATH_INFO: `/v3/users/0c8cdbb5b60b478a9155cbad581e6faa` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:55.915051 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea46a1f5-00aa-4c04-b240-a0d73f154c05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c8cdbb5b60b478a9155cbad581e6faa)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:55.917362 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea46a1f5-00aa-4c04-b240-a0d73f154c05 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:55.920207 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:56.017528 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ea46a1f5-00aa-4c04-b240-a0d73f154c05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c8cdbb5b60b478a9155cbad581e6faa'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:56.029616 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ea46a1f5-00aa-4c04-b240-a0d73f154c05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c8cdbb5b60b478a9155cbad581e6faa'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:56.060791 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ea46a1f5-00aa-4c04-b240-a0d73f154c05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c8cdbb5b60b478a9155cbad581e6faa'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:56.062105 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ea46a1f5-00aa-4c04-b240-a0d73f154c05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c8cdbb5b60b478a9155cbad581e6faa'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:56.074813 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ea46a1f5-00aa-4c04-b240-a0d73f154c05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c8cdbb5b60b478a9155cbad581e6faa'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:56.092162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1288/2591] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:15:55 2022] DELETE /identity/v3/users/0c8cdbb5b60b478a9155cbad581e6faa => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:15:56.097968 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d8924-d0f4-40d1-ba32-c2ae96f16284 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:56.098273 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d8924-d0f4-40d1-ba32-c2ae96f16284 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:56.098552 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d8924-d0f4-40d1-ba32-c2ae96f16284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:56.133291 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-676d8924-d0f4-40d1-ba32-c2ae96f16284 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:15:56.135301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-676d8924-d0f4-40d1-ba32-c2ae96f16284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:56.236636 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1304/2592] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:15:56.252476 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea46a1f5-00aa-4c04-b240-a0d73f154c05 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:56.255063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-ea46a1f5-00aa-4c04-b240-a0d73f154c05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:56.358610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea46a1f5-00aa-4c04-b240-a0d73f154c05 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:56.363296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f6a85-ca40-4a2e-9c0d-9d99d8f5c21c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:56.363925 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f6a85-ca40-4a2e-9c0d-9d99d8f5c21c service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:56.364246 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f6a85-ca40-4a2e-9c0d-9d99d8f5c21c service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:56.364517 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f6a85-ca40-4a2e-9c0d-9d99d8f5c21c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:56.374979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218f6a85-ca40-4a2e-9c0d-9d99d8f5c21c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:56.377536 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218f6a85-ca40-4a2e-9c0d-9d99d8f5c21c service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:56.389385 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1289/2593] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:15:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:15:56.668951 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d8924-d0f4-40d1-ba32-c2ae96f16284 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:56.671122 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-676d8924-d0f4-40d1-ba32-c2ae96f16284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:56.799190 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d8924-d0f4-40d1-ba32-c2ae96f16284 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:56.804889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d5a80-d3f4-4fe4-9920-197203a66e33 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61e2c954f9a49abad9f3de17ee39f63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:56.806795 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d5a80-d3f4-4fe4-9920-197203a66e33 service placement] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:56.806795 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d5a80-d3f4-4fe4-9920-197203a66e33 service placement] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:56.806795 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d5a80-d3f4-4fe4-9920-197203a66e33 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:56.808915 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-aa6d5a80-d3f4-4fe4-9920-197203a66e33 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:56.832594 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6d5a80-d3f4-4fe4-9920-197203a66e33 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:56.835401 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6d5a80-d3f4-4fe4-9920-197203a66e33 service placement] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:56.858087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f6a85-ca40-4a2e-9c0d-9d99d8f5c21c service neutron] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:56.864991 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-218f6a85-ca40-4a2e-9c0d-9d99d8f5c21c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:56.881463 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1305/2594] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:15:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:15:56.904093 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f6a85-ca40-4a2e-9c0d-9d99d8f5c21c service neutron] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:56.911535 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a1c68-ece6-44bc-b403-fa3489d63df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:56.915154 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a1c68-ece6-44bc-b403-fa3489d63df5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:56.915154 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a1c68-ece6-44bc-b403-fa3489d63df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:56.915154 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a1c68-ece6-44bc-b403-fa3489d63df5 admin admin] PATH_INFO: `/v3/projects/e0b1916130814fa4a2462683e362cac4` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:56.915857 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a1c68-ece6-44bc-b403-fa3489d63df5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0b1916130814fa4a2462683e362cac4)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:56.917947 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a1c68-ece6-44bc-b403-fa3489d63df5 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:57.002546 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-026a1c68-ece6-44bc-b403-fa3489d63df5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0b1916130814fa4a2462683e362cac4', 'request_id': 'req-026a1c68-ece6-44bc-b403-fa3489d63df5'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:57.017820 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-026a1c68-ece6-44bc-b403-fa3489d63df5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0b1916130814fa4a2462683e362cac4', 'request_id': 'req-026a1c68-ece6-44bc-b403-fa3489d63df5'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:57.042383 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-026a1c68-ece6-44bc-b403-fa3489d63df5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0b1916130814fa4a2462683e362cac4', 'request_id': 'req-026a1c68-ece6-44bc-b403-fa3489d63df5'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:57.068197 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-026a1c68-ece6-44bc-b403-fa3489d63df5 admin admin] The token cache is being invalidate because project e0b1916130814fa4a2462683e362cac4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:15:57.072899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-026a1c68-ece6-44bc-b403-fa3489d63df5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:57.074337 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-026a1c68-ece6-44bc-b403-fa3489d63df5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:57.077763 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1290/2595] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:15:56 2022] DELETE /identity/v3/projects/e0b1916130814fa4a2462683e362cac4 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:15:57.084699 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d5a80-d3f4-4fe4-9920-197203a66e33 service placement] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:57.111109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d5a80-d3f4-4fe4-9920-197203a66e33 service placement] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:57.193611 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:57.214436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e9252-728c-4bc5-a539-1ba9c4e3a62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:57.215109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e9252-728c-4bc5-a539-1ba9c4e3a62e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:57.215463 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e9252-728c-4bc5-a539-1ba9c4e3a62e admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:57.215808 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e9252-728c-4bc5-a539-1ba9c4e3a62e admin admin] PATH_INFO: `/v3/users/7f955e9ee1704b39b570e83d30450267` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:57.217292 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e9252-728c-4bc5-a539-1ba9c4e3a62e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f955e9ee1704b39b570e83d30450267)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:57.220773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e9252-728c-4bc5-a539-1ba9c4e3a62e admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:57.298855 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-883e9252-728c-4bc5-a539-1ba9c4e3a62e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f955e9ee1704b39b570e83d30450267'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:57.310640 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-883e9252-728c-4bc5-a539-1ba9c4e3a62e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f955e9ee1704b39b570e83d30450267'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:57.324383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-883e9252-728c-4bc5-a539-1ba9c4e3a62e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f955e9ee1704b39b570e83d30450267'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:57.347247 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-883e9252-728c-4bc5-a539-1ba9c4e3a62e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f955e9ee1704b39b570e83d30450267'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:57.349046 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-883e9252-728c-4bc5-a539-1ba9c4e3a62e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f955e9ee1704b39b570e83d30450267'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:57.361131 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1306/2596] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:15:57 2022] DELETE /identity/v3/users/7f955e9ee1704b39b570e83d30450267 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:15:57.893543 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a1c68-ece6-44bc-b403-fa3489d63df5 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:57.896937 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-026a1c68-ece6-44bc-b403-fa3489d63df5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:57.922001 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a1c68-ece6-44bc-b403-fa3489d63df5 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:57.926581 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:58.016428 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bfe0c4-df35-49b4-a695-0ad662630cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:58.017461 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfe0c4-df35-49b4-a695-0ad662630cc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:58.017836 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfe0c4-df35-49b4-a695-0ad662630cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:58.018189 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfe0c4-df35-49b4-a695-0ad662630cc6 admin admin] PATH_INFO: `/v3/projects/f70812d542a748f982a7cd3e2c203ebe` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:58.019788 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bfe0c4-df35-49b4-a695-0ad662630cc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f70812d542a748f982a7cd3e2c203ebe)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:58.022004 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bfe0c4-df35-49b4-a695-0ad662630cc6 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:58.085477 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-d1bfe0c4-df35-49b4-a695-0ad662630cc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f70812d542a748f982a7cd3e2c203ebe', 'request_id': 'req-d1bfe0c4-df35-49b4-a695-0ad662630cc6'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:58.099635 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-d1bfe0c4-df35-49b4-a695-0ad662630cc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f70812d542a748f982a7cd3e2c203ebe', 'request_id': 'req-d1bfe0c4-df35-49b4-a695-0ad662630cc6'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:58.110145 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-d1bfe0c4-df35-49b4-a695-0ad662630cc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f70812d542a748f982a7cd3e2c203ebe', 'request_id': 'req-d1bfe0c4-df35-49b4-a695-0ad662630cc6'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:58.132029 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-d1bfe0c4-df35-49b4-a695-0ad662630cc6 admin admin] The token cache is being invalidate because project f70812d542a748f982a7cd3e2c203ebe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:15:58.132477 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-d1bfe0c4-df35-49b4-a695-0ad662630cc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:58.133271 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-d1bfe0c4-df35-49b4-a695-0ad662630cc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:15:58.134883 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1291/2597] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:15:57 2022] DELETE /identity/v3/projects/f70812d542a748f982a7cd3e2c203ebe => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:15:58.139864 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f93470-1539-4fef-bc9c-d5268b02d7cc None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:58.140368 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f93470-1539-4fef-bc9c-d5268b02d7cc None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:58.140474 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f93470-1539-4fef-bc9c-d5268b02d7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:58.180564 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-e2f93470-1539-4fef-bc9c-d5268b02d7cc None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:15:58.180564 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e2f93470-1539-4fef-bc9c-d5268b02d7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:58.263068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1307/2598] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:15:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:15:58.336970 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c151de-fc08-4aad-94a6-f4bd7c62eb6f None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:58.337923 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c151de-fc08-4aad-94a6-f4bd7c62eb6f None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:58.338888 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c151de-fc08-4aad-94a6-f4bd7c62eb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:58.370410 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-e8c151de-fc08-4aad-94a6-f4bd7c62eb6f None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:15:58.372625 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-e8c151de-fc08-4aad-94a6-f4bd7c62eb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:58.388898 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1292/2599] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:15:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:15:58.390237 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f93470-1539-4fef-bc9c-d5268b02d7cc None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:58.401986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f93470-1539-4fef-bc9c-d5268b02d7cc None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:58.421855 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ffe959-c101-4323-8ab2-d615a632d80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:58.423622 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffe959-c101-4323-8ab2-d615a632d80e admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:58.424064 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffe959-c101-4323-8ab2-d615a632d80e admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:58.424496 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffe959-c101-4323-8ab2-d615a632d80e admin admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:58.425687 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ffe959-c101-4323-8ab2-d615a632d80e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:15:58.426146 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ffe959-c101-4323-8ab2-d615a632d80e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:58.428804 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ffe959-c101-4323-8ab2-d615a632d80e admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:58.451303 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1308/2600] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:15:58.457039 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c151de-fc08-4aad-94a6-f4bd7c62eb6f None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:58.483665 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c151de-fc08-4aad-94a6-f4bd7c62eb6f None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:58.509672 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0facd91-5fee-488e-a86e-75c82928fc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:58.511304 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0facd91-5fee-488e-a86e-75c82928fc50 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:58.511304 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0facd91-5fee-488e-a86e-75c82928fc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:58.511710 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0facd91-5fee-488e-a86e-75c82928fc50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:58.512892 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0facd91-5fee-488e-a86e-75c82928fc50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:58.515043 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0facd91-5fee-488e-a86e-75c82928fc50 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:58.564220 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1293/2601] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:15:58 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:15:58.567304 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ffe959-c101-4323-8ab2-d615a632d80e admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:58.578785 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ffe959-c101-4323-8ab2-d615a632d80e admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:58.654718 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:58.675255 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdafcf19-e65d-4479-8acf-03863b16cdf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:58.675891 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdafcf19-e65d-4479-8acf-03863b16cdf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:58.676172 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdafcf19-e65d-4479-8acf-03863b16cdf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:58.676444 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdafcf19-e65d-4479-8acf-03863b16cdf9 admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:58.677339 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdafcf19-e65d-4479-8acf-03863b16cdf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:58.679334 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdafcf19-e65d-4479-8acf-03863b16cdf9 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:58.705353 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1309/2602] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:15:58 2022] POST /identity/v3/users => generated 293 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:15:58.710295 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0facd91-5fee-488e-a86e-75c82928fc50 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:58.739169 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0facd91-5fee-488e-a86e-75c82928fc50 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:58.754102 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2efdf97-9c7b-43c1-a342-f86a2cf48a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:58.754460 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efdf97-9c7b-43c1-a342-f86a2cf48a67 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:58.755036 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efdf97-9c7b-43c1-a342-f86a2cf48a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:58.755579 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efdf97-9c7b-43c1-a342-f86a2cf48a67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:58.756943 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2efdf97-9c7b-43c1-a342-f86a2cf48a67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:58.759757 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2efdf97-9c7b-43c1-a342-f86a2cf48a67 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:58.775924 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1294/2603] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:15:58.780503 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdafcf19-e65d-4479-8acf-03863b16cdf9 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:58.794869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdafcf19-e65d-4479-8acf-03863b16cdf9 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:58.802054 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f05f0-bebd-4a75-a78d-a85f9595e9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:58.802919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f05f0-bebd-4a75-a78d-a85f9595e9c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:58.803272 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f05f0-bebd-4a75-a78d-a85f9595e9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:58.803612 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f05f0-bebd-4a75-a78d-a85f9595e9c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:58.804384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f05f0-bebd-4a75-a78d-a85f9595e9c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:58.806099 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f05f0-bebd-4a75-a78d-a85f9595e9c5 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:58.822917 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1310/2604] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:15:58.827120 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2efdf97-9c7b-43c1-a342-f86a2cf48a67 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:58.838606 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2efdf97-9c7b-43c1-a342-f86a2cf48a67 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:58.844319 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b3434-e4dd-498f-a798-d0a50d1838dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:58.845061 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b3434-e4dd-498f-a798-d0a50d1838dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:58.845342 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b3434-e4dd-498f-a798-d0a50d1838dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:58.845618 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b3434-e4dd-498f-a798-d0a50d1838dc admin admin] PATH_INFO: `/v3/projects/86983b89b4404c8e82470dc5227d541b/users/d2e71ee95f0c4c098a27d24ad6ebaf11/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:58.886110 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675b3434-e4dd-498f-a798-d0a50d1838dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=86983b89b4404c8e82470dc5227d541b, user_id=d2e71ee95f0c4c098a27d24ad6ebaf11, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:58.893884 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675b3434-e4dd-498f-a798-d0a50d1838dc admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:58.931705 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1295/2605] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:15:58 2022] PUT /identity/v3/projects/86983b89b4404c8e82470dc5227d541b/users/d2e71ee95f0c4c098a27d24ad6ebaf11/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:15:58.942403 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158e949-2a69-404a-bf8e-ac612c0dcc22 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:58.942858 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158e949-2a69-404a-bf8e-ac612c0dcc22 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:58.943302 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158e949-2a69-404a-bf8e-ac612c0dcc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:58.978587 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-9158e949-2a69-404a-bf8e-ac612c0dcc22 None None] MFA Rules not processed for user `d2e71ee95f0c4c098a27d24ad6ebaf11`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:15:58.979902 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-9158e949-2a69-404a-bf8e-ac612c0dcc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:59.111180 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1311/2606] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:15:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:15:59.116849 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b3434-e4dd-498f-a798-d0a50d1838dc admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:59.167906 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:15:59.206853 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b3434-e4dd-498f-a798-d0a50d1838dc admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:59.211779 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21fbb4a-7dba-428d-b907-d189319bacaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:59.212468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21fbb4a-7dba-428d-b907-d189319bacaa admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:59.212848 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21fbb4a-7dba-428d-b907-d189319bacaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:59.213394 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21fbb4a-7dba-428d-b907-d189319bacaa admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:59.214166 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21fbb4a-7dba-428d-b907-d189319bacaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:59.216070 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21fbb4a-7dba-428d-b907-d189319bacaa admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:59.247095 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1296/2607] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:15:59 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:15:59.252522 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158e949-2a69-404a-bf8e-ac612c0dcc22 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:59.264987 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158e949-2a69-404a-bf8e-ac612c0dcc22 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:59.332991 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b86784b-0c6f-4778-91a1-75812ba5ca1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:59.333533 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86784b-0c6f-4778-91a1-75812ba5ca1f admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:59.333837 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86784b-0c6f-4778-91a1-75812ba5ca1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:59.334431 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86784b-0c6f-4778-91a1-75812ba5ca1f admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:59.334985 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b86784b-0c6f-4778-91a1-75812ba5ca1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:59.336992 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b86784b-0c6f-4778-91a1-75812ba5ca1f admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:59.370825 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1312/2608] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:15:59 2022] POST /identity/v3/users => generated 299 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:15:59.372605 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21fbb4a-7dba-428d-b907-d189319bacaa admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:59.398032 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21fbb4a-7dba-428d-b907-d189319bacaa admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:59.405741 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358f5287-d93d-4577-a0a6-44bbf7fa8244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:59.407258 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f5287-d93d-4577-a0a6-44bbf7fa8244 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:59.407699 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f5287-d93d-4577-a0a6-44bbf7fa8244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:59.408131 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f5287-d93d-4577-a0a6-44bbf7fa8244 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:59.409099 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358f5287-d93d-4577-a0a6-44bbf7fa8244 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:59.410891 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358f5287-d93d-4577-a0a6-44bbf7fa8244 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:59.428174 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1297/2609] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:15:59.433666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b86784b-0c6f-4778-91a1-75812ba5ca1f admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:59.448766 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b86784b-0c6f-4778-91a1-75812ba5ca1f admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:59.479628 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaaba53-bd44-4d11-85d9-2ca6744a3b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:59.480593 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaaba53-bd44-4d11-85d9-2ca6744a3b6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:59.480952 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaaba53-bd44-4d11-85d9-2ca6744a3b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:59.481305 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaaba53-bd44-4d11-85d9-2ca6744a3b6a admin admin] PATH_INFO: `/v3/projects/775b55c0a2eb4fb6b0a81d94e500e939/users/504b7fe3acbd42d3ac760aae3fba61a9/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:59.500880 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaaba53-bd44-4d11-85d9-2ca6744a3b6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=775b55c0a2eb4fb6b0a81d94e500e939, user_id=504b7fe3acbd42d3ac760aae3fba61a9, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:59.503448 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaaba53-bd44-4d11-85d9-2ca6744a3b6a admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:59.525034 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1313/2610] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:15:59 2022] PUT /identity/v3/projects/775b55c0a2eb4fb6b0a81d94e500e939/users/504b7fe3acbd42d3ac760aae3fba61a9/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:15:59.528882 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358f5287-d93d-4577-a0a6-44bbf7fa8244 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:59.549068 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358f5287-d93d-4577-a0a6-44bbf7fa8244 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:59.646295 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32756111-190b-42a3-8c89-3313cb6c628d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:59.647688 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32756111-190b-42a3-8c89-3313cb6c628d admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:59.648151 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32756111-190b-42a3-8c89-3313cb6c628d admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:59.648569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32756111-190b-42a3-8c89-3313cb6c628d admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:59.649557 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32756111-190b-42a3-8c89-3313cb6c628d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:59.651676 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32756111-190b-42a3-8c89-3313cb6c628d admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:59.668971 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1298/2611] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:15:59.670400 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaaba53-bd44-4d11-85d9-2ca6744a3b6a admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:59.691812 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaaba53-bd44-4d11-85d9-2ca6744a3b6a admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:15:59.699629 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c4049-fac5-49a5-90ff-e408f978496c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:15:59.700985 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c4049-fac5-49a5-90ff-e408f978496c admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:59.701496 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c4049-fac5-49a5-90ff-e408f978496c admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:59.701989 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c4049-fac5-49a5-90ff-e408f978496c admin admin] PATH_INFO: `/v3/projects/775b55c0a2eb4fb6b0a81d94e500e939/users/504b7fe3acbd42d3ac760aae3fba61a9/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:59.706301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c4049-fac5-49a5-90ff-e408f978496c admin admin] RBAC: Authorizing `identity:create_grant(project_id=775b55c0a2eb4fb6b0a81d94e500e939, user_id=504b7fe3acbd42d3ac760aae3fba61a9, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:15:59.711929 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c4049-fac5-49a5-90ff-e408f978496c admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:15:59.728755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1314/2612] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:15:59 2022] PUT /identity/v3/projects/775b55c0a2eb4fb6b0a81d94e500e939/users/504b7fe3acbd42d3ac760aae3fba61a9/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:15:59.747118 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710e0e6-ca62-4caf-9eff-94c29940a6db None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:15:59.747938 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710e0e6-ca62-4caf-9eff-94c29940a6db None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:15:59.748387 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710e0e6-ca62-4caf-9eff-94c29940a6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:15:59.837222 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-6710e0e6-ca62-4caf-9eff-94c29940a6db None None] MFA Rules not processed for user `504b7fe3acbd42d3ac760aae3fba61a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:15:59.840486 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-6710e0e6-ca62-4caf-9eff-94c29940a6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:15:59.960650 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1299/2613] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:15:59 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:15:59.979259 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c4049-fac5-49a5-90ff-e408f978496c admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:15:59.989497 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e68c4049-fac5-49a5-90ff-e408f978496c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:00.007695 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:00.083600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c4049-fac5-49a5-90ff-e408f978496c admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:00.088191 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f772c83-dd27-4a88-9664-a21a882f22c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:00.089276 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f772c83-dd27-4a88-9664-a21a882f22c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:00.089639 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f772c83-dd27-4a88-9664-a21a882f22c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:00.089966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f772c83-dd27-4a88-9664-a21a882f22c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:00.181772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f772c83-dd27-4a88-9664-a21a882f22c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:00.181772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f772c83-dd27-4a88-9664-a21a882f22c8 service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:00.197854 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1315/2614] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:15:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:16:00.534149 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710e0e6-ca62-4caf-9eff-94c29940a6db None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:00.536011 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-6710e0e6-ca62-4caf-9eff-94c29940a6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:00.594484 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:00.646188 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710e0e6-ca62-4caf-9eff-94c29940a6db None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:00.655397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afbef4-7198-4f92-bbc3-b326cf913c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:00.656147 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afbef4-7198-4f92-bbc3-b326cf913c3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:00.657631 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afbef4-7198-4f92-bbc3-b326cf913c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:00.658351 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afbef4-7198-4f92-bbc3-b326cf913c3e admin admin] PATH_INFO: `/v3/users/160c88a7ba0144d4b1bc953767b12bb7` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:00.660409 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62afbef4-7198-4f92-bbc3-b326cf913c3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=160c88a7ba0144d4b1bc953767b12bb7)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:00.664696 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62afbef4-7198-4f92-bbc3-b326cf913c3e admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:00.734087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-62afbef4-7198-4f92-bbc3-b326cf913c3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '160c88a7ba0144d4b1bc953767b12bb7'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:00.746809 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-62afbef4-7198-4f92-bbc3-b326cf913c3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '160c88a7ba0144d4b1bc953767b12bb7'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:00.767226 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-62afbef4-7198-4f92-bbc3-b326cf913c3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '160c88a7ba0144d4b1bc953767b12bb7'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:00.767916 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-62afbef4-7198-4f92-bbc3-b326cf913c3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '160c88a7ba0144d4b1bc953767b12bb7'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:00.778407 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-62afbef4-7198-4f92-bbc3-b326cf913c3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '160c88a7ba0144d4b1bc953767b12bb7'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:00.786850 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1300/2615] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:16:00 2022] DELETE /identity/v3/users/160c88a7ba0144d4b1bc953767b12bb7 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:16:01.005054 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f772c83-dd27-4a88-9664-a21a882f22c8 service cinder] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:01.024469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f772c83-dd27-4a88-9664-a21a882f22c8 service cinder] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:01.120079 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5e886-0e67-41ab-9304-ae6575cc1fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:01.120845 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5e886-0e67-41ab-9304-ae6575cc1fc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:01.121311 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5e886-0e67-41ab-9304-ae6575cc1fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:01.121664 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5e886-0e67-41ab-9304-ae6575cc1fc8 admin admin] PATH_INFO: `/v3/projects/aa4789b47e0f47b0acd1cc4919593c33` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:01.122936 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e5e886-0e67-41ab-9304-ae6575cc1fc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa4789b47e0f47b0acd1cc4919593c33)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:01.125012 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e5e886-0e67-41ab-9304-ae6575cc1fc8 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:01.186940 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-c8e5e886-0e67-41ab-9304-ae6575cc1fc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa4789b47e0f47b0acd1cc4919593c33', 'request_id': 'req-c8e5e886-0e67-41ab-9304-ae6575cc1fc8'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:01.194859 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-c8e5e886-0e67-41ab-9304-ae6575cc1fc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa4789b47e0f47b0acd1cc4919593c33', 'request_id': 'req-c8e5e886-0e67-41ab-9304-ae6575cc1fc8'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:01.204140 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-c8e5e886-0e67-41ab-9304-ae6575cc1fc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa4789b47e0f47b0acd1cc4919593c33', 'request_id': 'req-c8e5e886-0e67-41ab-9304-ae6575cc1fc8'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:01.209196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:01.221463 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-c8e5e886-0e67-41ab-9304-ae6575cc1fc8 admin admin] The token cache is being invalidate because project aa4789b47e0f47b0acd1cc4919593c33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:16:01.221845 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-c8e5e886-0e67-41ab-9304-ae6575cc1fc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:01.222534 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-c8e5e886-0e67-41ab-9304-ae6575cc1fc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:01.224513 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1316/2616] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:16:01 2022] DELETE /identity/v3/projects/aa4789b47e0f47b0acd1cc4919593c33 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:16:01.229642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cc4f2-ae36-4f7f-bbc1-0b54d66ea654 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:01.230407 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cc4f2-ae36-4f7f-bbc1-0b54d66ea654 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:01.230811 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cc4f2-ae36-4f7f-bbc1-0b54d66ea654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:01.260430 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-903cc4f2-ae36-4f7f-bbc1-0b54d66ea654 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:01.261797 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-903cc4f2-ae36-4f7f-bbc1-0b54d66ea654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:01.340889 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1301/2617] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:16:01.349267 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a0ce6-887a-4869-8119-69cccfb4d22c None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:01.349558 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a0ce6-887a-4869-8119-69cccfb4d22c None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:01.349844 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a0ce6-887a-4869-8119-69cccfb4d22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:01.370996 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-720a0ce6-887a-4869-8119-69cccfb4d22c None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:01.372225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-720a0ce6-887a-4869-8119-69cccfb4d22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:01.382766 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1317/2618] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:16:01.387912 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cc4f2-ae36-4f7f-bbc1-0b54d66ea654 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:01.401401 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cc4f2-ae36-4f7f-bbc1-0b54d66ea654 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:01.407787 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bded5d-e819-4984-a9fa-02459fe71d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:01.408680 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bded5d-e819-4984-a9fa-02459fe71d26 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:01.408962 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bded5d-e819-4984-a9fa-02459fe71d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:01.409234 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bded5d-e819-4984-a9fa-02459fe71d26 admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:01.409937 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bded5d-e819-4984-a9fa-02459fe71d26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:16:01.410298 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bded5d-e819-4984-a9fa-02459fe71d26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:01.414239 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bded5d-e819-4984-a9fa-02459fe71d26 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:01.426212 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1302/2619] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:16:01.431134 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a0ce6-887a-4869-8119-69cccfb4d22c None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:01.442265 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a0ce6-887a-4869-8119-69cccfb4d22c None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:01.447250 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988fc1e0-1e7d-4f3b-9516-a16b2a2dea32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:01.447854 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988fc1e0-1e7d-4f3b-9516-a16b2a2dea32 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:01.448152 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988fc1e0-1e7d-4f3b-9516-a16b2a2dea32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:01.448861 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988fc1e0-1e7d-4f3b-9516-a16b2a2dea32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:01.449106 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988fc1e0-1e7d-4f3b-9516-a16b2a2dea32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:01.451009 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988fc1e0-1e7d-4f3b-9516-a16b2a2dea32 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:01.489362 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1318/2620] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:16:01 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:16:01.491623 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bded5d-e819-4984-a9fa-02459fe71d26 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:01.516620 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bded5d-e819-4984-a9fa-02459fe71d26 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:01.588642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:01.624016 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5bdce2-c252-4a15-8302-47cac991260d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:01.625634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5bdce2-c252-4a15-8302-47cac991260d admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:01.625634 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5bdce2-c252-4a15-8302-47cac991260d admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:01.625861 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5bdce2-c252-4a15-8302-47cac991260d admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:01.627268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5bdce2-c252-4a15-8302-47cac991260d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:01.628921 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5bdce2-c252-4a15-8302-47cac991260d admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:01.667505 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988fc1e0-1e7d-4f3b-9516-a16b2a2dea32 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:01.678538 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1303/2621] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:16:01 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:16:01.683436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988fc1e0-1e7d-4f3b-9516-a16b2a2dea32 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:01.691960 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4f7c1-0780-4d49-8174-34cfbee475bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:01.692940 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4f7c1-0780-4d49-8174-34cfbee475bd admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:01.693429 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4f7c1-0780-4d49-8174-34cfbee475bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:01.693844 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4f7c1-0780-4d49-8174-34cfbee475bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:01.694727 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a4f7c1-0780-4d49-8174-34cfbee475bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:01.696537 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a4f7c1-0780-4d49-8174-34cfbee475bd admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:01.727272 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1319/2622] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:16:01.731265 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5bdce2-c252-4a15-8302-47cac991260d admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:01.742469 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5bdce2-c252-4a15-8302-47cac991260d admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:01.747651 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd18be-ebc3-43f3-b98c-2faf0bce5960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:01.748275 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd18be-ebc3-43f3-b98c-2faf0bce5960 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:01.748638 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd18be-ebc3-43f3-b98c-2faf0bce5960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:01.748929 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd18be-ebc3-43f3-b98c-2faf0bce5960 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:01.749616 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dd18be-ebc3-43f3-b98c-2faf0bce5960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:01.752447 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dd18be-ebc3-43f3-b98c-2faf0bce5960 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:01.764269 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1304/2623] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:16:01.769880 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4f7c1-0780-4d49-8174-34cfbee475bd admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:01.785782 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4f7c1-0780-4d49-8174-34cfbee475bd admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:01.794090 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40cb311-4de1-4113-a0e9-674dca7d4dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:01.794878 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40cb311-4de1-4113-a0e9-674dca7d4dfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:01.795335 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40cb311-4de1-4113-a0e9-674dca7d4dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:01.795605 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40cb311-4de1-4113-a0e9-674dca7d4dfe admin admin] PATH_INFO: `/v3/projects/03344da5c61d404a9361b4f9aeed1cd4/users/980873d89ec84c8284eb5e78a42b072f/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:01.820900 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40cb311-4de1-4113-a0e9-674dca7d4dfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=03344da5c61d404a9361b4f9aeed1cd4, user_id=980873d89ec84c8284eb5e78a42b072f, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:01.827021 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40cb311-4de1-4113-a0e9-674dca7d4dfe admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:01.864684 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1320/2624] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:16:01 2022] PUT /identity/v3/projects/03344da5c61d404a9361b4f9aeed1cd4/users/980873d89ec84c8284eb5e78a42b072f/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:16:01.870351 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd23a8-bee5-4dcb-96b1-e833437d0d85 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:01.870643 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd23a8-bee5-4dcb-96b1-e833437d0d85 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:01.870926 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd23a8-bee5-4dcb-96b1-e833437d0d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:01.915418 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-7afd23a8-bee5-4dcb-96b1-e833437d0d85 None None] MFA Rules not processed for user `980873d89ec84c8284eb5e78a42b072f`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:01.922501 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-7afd23a8-bee5-4dcb-96b1-e833437d0d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:02.015577 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1305/2625] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:16:02.027407 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40cb311-4de1-4113-a0e9-674dca7d4dfe admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:02.029640 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-f40cb311-4de1-4113-a0e9-674dca7d4dfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:02.146602 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40cb311-4de1-4113-a0e9-674dca7d4dfe admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:02.154478 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b8dc0-9df2-431b-ae03-81b1a74ff856 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:02.155265 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b8dc0-9df2-431b-ae03-81b1a74ff856 service nova] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:02.155585 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b8dc0-9df2-431b-ae03-81b1a74ff856 service nova] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:02.155854 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b8dc0-9df2-431b-ae03-81b1a74ff856 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:02.167210 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0b8dc0-9df2-431b-ae03-81b1a74ff856 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:02.170347 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0b8dc0-9df2-431b-ae03-81b1a74ff856 service nova] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:02.187289 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1321/2626] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:16:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:16:04.821064 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd23a8-bee5-4dcb-96b1-e833437d0d85 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:04.822493 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-7afd23a8-bee5-4dcb-96b1-e833437d0d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:04.890260 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd23a8-bee5-4dcb-96b1-e833437d0d85 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:04.895520 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779018b-98a7-4161-8046-2ceac7dc54e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:04.896223 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779018b-98a7-4161-8046-2ceac7dc54e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:04.896517 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779018b-98a7-4161-8046-2ceac7dc54e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:04.896852 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779018b-98a7-4161-8046-2ceac7dc54e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:04.898728 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-2779018b-98a7-4161-8046-2ceac7dc54e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:04.966165 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:04.970809 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779018b-98a7-4161-8046-2ceac7dc54e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:04.973032 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779018b-98a7-4161-8046-2ceac7dc54e6 service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:04.983798 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1306/2627] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:16:04 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:16:11.955429 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b8dc0-9df2-431b-ae03-81b1a74ff856 service nova] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:12.026413 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b8dc0-9df2-431b-ae03-81b1a74ff856 service nova] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:12.032265 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2881ad-d20a-4066-a0ff-7d4eabcc7289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:12.032959 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2881ad-d20a-4066-a0ff-7d4eabcc7289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:12.033344 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2881ad-d20a-4066-a0ff-7d4eabcc7289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:12.033621 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2881ad-d20a-4066-a0ff-7d4eabcc7289 admin admin] PATH_INFO: `/v3/users/980873d89ec84c8284eb5e78a42b072f` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:12.034990 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2881ad-d20a-4066-a0ff-7d4eabcc7289 admin admin] RBAC: Authorizing `identity:delete_user(user_id=980873d89ec84c8284eb5e78a42b072f)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:12.037022 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2881ad-d20a-4066-a0ff-7d4eabcc7289 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:12.064234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:12.106276 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-2f2881ad-d20a-4066-a0ff-7d4eabcc7289 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '980873d89ec84c8284eb5e78a42b072f'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:12.118104 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-2f2881ad-d20a-4066-a0ff-7d4eabcc7289 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '980873d89ec84c8284eb5e78a42b072f'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:12.129663 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-2f2881ad-d20a-4066-a0ff-7d4eabcc7289 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '980873d89ec84c8284eb5e78a42b072f'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:12.154220 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-2f2881ad-d20a-4066-a0ff-7d4eabcc7289 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '980873d89ec84c8284eb5e78a42b072f'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:12.155444 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-2f2881ad-d20a-4066-a0ff-7d4eabcc7289 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '980873d89ec84c8284eb5e78a42b072f'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:12.164877 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1322/2628] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:16:11 2022] DELETE /identity/v3/users/980873d89ec84c8284eb5e78a42b072f => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:16:12.173349 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80afa2d-73ea-403d-ae00-3ce1e7714ea3 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:12.174034 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80afa2d-73ea-403d-ae00-3ce1e7714ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:12.174398 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80afa2d-73ea-403d-ae00-3ce1e7714ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:12.202646 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-e80afa2d-73ea-403d-ae00-3ce1e7714ea3 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:12.205417 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-e80afa2d-73ea-403d-ae00-3ce1e7714ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:12.299711 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1307/2629] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:16:12.310870 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2881ad-d20a-4066-a0ff-7d4eabcc7289 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:12.312479 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-2f2881ad-d20a-4066-a0ff-7d4eabcc7289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:12.377912 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2881ad-d20a-4066-a0ff-7d4eabcc7289 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:12.381876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b740c-107b-4d57-852f-09a923173e01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:12.382606 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b740c-107b-4d57-852f-09a923173e01 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:12.382893 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b740c-107b-4d57-852f-09a923173e01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:12.383433 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b740c-107b-4d57-852f-09a923173e01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:12.396251 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3b740c-107b-4d57-852f-09a923173e01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:12.397768 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3b740c-107b-4d57-852f-09a923173e01 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:12.408846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1323/2630] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:16:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:16:12.763565 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80afa2d-73ea-403d-ae00-3ce1e7714ea3 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:12.763835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-e80afa2d-73ea-403d-ae00-3ce1e7714ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:12.786355 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80afa2d-73ea-403d-ae00-3ce1e7714ea3 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:12.794246 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065359c-5c95-435c-8e73-39da3ea418b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:12.794924 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065359c-5c95-435c-8e73-39da3ea418b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:12.795449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065359c-5c95-435c-8e73-39da3ea418b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:12.795867 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065359c-5c95-435c-8e73-39da3ea418b9 admin admin] PATH_INFO: `/v3/projects/03344da5c61d404a9361b4f9aeed1cd4` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:12.798757 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065359c-5c95-435c-8e73-39da3ea418b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03344da5c61d404a9361b4f9aeed1cd4)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:12.802175 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065359c-5c95-435c-8e73-39da3ea418b9 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:12.895804 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-6065359c-5c95-435c-8e73-39da3ea418b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03344da5c61d404a9361b4f9aeed1cd4', 'request_id': 'req-6065359c-5c95-435c-8e73-39da3ea418b9'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:12.909709 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-6065359c-5c95-435c-8e73-39da3ea418b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03344da5c61d404a9361b4f9aeed1cd4', 'request_id': 'req-6065359c-5c95-435c-8e73-39da3ea418b9'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:12.918933 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-6065359c-5c95-435c-8e73-39da3ea418b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03344da5c61d404a9361b4f9aeed1cd4', 'request_id': 'req-6065359c-5c95-435c-8e73-39da3ea418b9'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:12.932570 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-6065359c-5c95-435c-8e73-39da3ea418b9 admin admin] The token cache is being invalidate because project 03344da5c61d404a9361b4f9aeed1cd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:16:12.932896 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-6065359c-5c95-435c-8e73-39da3ea418b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:12.933541 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-6065359c-5c95-435c-8e73-39da3ea418b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:12.935301 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1308/2631] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:16:12 2022] DELETE /identity/v3/projects/03344da5c61d404a9361b4f9aeed1cd4 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:16:12.941047 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f372c0-1fb6-43c7-8d36-05579e0f46c7 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:12.941328 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f372c0-1fb6-43c7-8d36-05579e0f46c7 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:12.941607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f372c0-1fb6-43c7-8d36-05579e0f46c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:12.963781 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-86f372c0-1fb6-43c7-8d36-05579e0f46c7 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:12.965057 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-86f372c0-1fb6-43c7-8d36-05579e0f46c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:13.053962 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1324/2632] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:16:13.064085 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c2d61-129f-4c11-b35e-c206267092b0 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:13.064548 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c2d61-129f-4c11-b35e-c206267092b0 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:13.065038 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c2d61-129f-4c11-b35e-c206267092b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:13.096882 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-1f3c2d61-129f-4c11-b35e-c206267092b0 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:13.098792 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-1f3c2d61-129f-4c11-b35e-c206267092b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:13.112208 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1309/2633] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:16:13.118700 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f372c0-1fb6-43c7-8d36-05579e0f46c7 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:13.135782 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f372c0-1fb6-43c7-8d36-05579e0f46c7 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:13.145956 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c32183-c85a-410d-b9a8-bd9f005a1427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:13.147070 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c32183-c85a-410d-b9a8-bd9f005a1427 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:13.147567 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c32183-c85a-410d-b9a8-bd9f005a1427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:13.148057 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c32183-c85a-410d-b9a8-bd9f005a1427 admin admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:13.149436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c32183-c85a-410d-b9a8-bd9f005a1427 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:16:13.150020 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c32183-c85a-410d-b9a8-bd9f005a1427 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:13.153220 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c32183-c85a-410d-b9a8-bd9f005a1427 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:13.169932 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1325/2634] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:16:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:16:13.175545 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c2d61-129f-4c11-b35e-c206267092b0 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:13.186132 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c2d61-129f-4c11-b35e-c206267092b0 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:13.191478 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5394c5-be79-491d-becd-ed543d112722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:13.192212 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5394c5-be79-491d-becd-ed543d112722 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:13.192533 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5394c5-be79-491d-becd-ed543d112722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:13.192932 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5394c5-be79-491d-becd-ed543d112722 admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:13.193683 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5394c5-be79-491d-becd-ed543d112722 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:13.195791 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5394c5-be79-491d-becd-ed543d112722 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:13.233497 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1310/2635] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:16:13 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:16:13.240177 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c32183-c85a-410d-b9a8-bd9f005a1427 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:13.252264 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c32183-c85a-410d-b9a8-bd9f005a1427 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:13.282781 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:13.344956 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170d384-9747-4f7e-9b1f-509b6fa9abe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:13.344956 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170d384-9747-4f7e-9b1f-509b6fa9abe3 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:13.344956 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170d384-9747-4f7e-9b1f-509b6fa9abe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:13.344956 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170d384-9747-4f7e-9b1f-509b6fa9abe3 admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:13.345984 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3170d384-9747-4f7e-9b1f-509b6fa9abe3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:13.345984 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3170d384-9747-4f7e-9b1f-509b6fa9abe3 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:13.375999 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1326/2636] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:16:13 2022] POST /identity/v3/users => generated 287 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:16:13.382229 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5394c5-be79-491d-becd-ed543d112722 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:13.399021 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5394c5-be79-491d-becd-ed543d112722 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:13.408055 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe891ec0-e7bd-455a-ad72-36db286e1cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:13.409309 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe891ec0-e7bd-455a-ad72-36db286e1cb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:13.409771 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe891ec0-e7bd-455a-ad72-36db286e1cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:13.410254 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe891ec0-e7bd-455a-ad72-36db286e1cb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:13.411435 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe891ec0-e7bd-455a-ad72-36db286e1cb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:13.414953 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe891ec0-e7bd-455a-ad72-36db286e1cb9 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:13.434356 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1311/2637] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:16:13.441123 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170d384-9747-4f7e-9b1f-509b6fa9abe3 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:13.460854 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170d384-9747-4f7e-9b1f-509b6fa9abe3 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:13.467874 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ed650-f07a-495f-8844-ca67c70daacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:13.469943 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ed650-f07a-495f-8844-ca67c70daacf admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:13.471226 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ed650-f07a-495f-8844-ca67c70daacf admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:13.485204 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ed650-f07a-495f-8844-ca67c70daacf admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:13.485204 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470ed650-f07a-495f-8844-ca67c70daacf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:13.485204 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470ed650-f07a-495f-8844-ca67c70daacf admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:13.493529 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1327/2638] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:16:13.497733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe891ec0-e7bd-455a-ad72-36db286e1cb9 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:13.509940 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe891ec0-e7bd-455a-ad72-36db286e1cb9 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:13.515117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf3518-b6ef-450c-8230-e2bde40ed7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:13.516420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf3518-b6ef-450c-8230-e2bde40ed7c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:13.516759 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf3518-b6ef-450c-8230-e2bde40ed7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:13.517063 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf3518-b6ef-450c-8230-e2bde40ed7c1 admin admin] PATH_INFO: `/v3/projects/985012e7b45949f5ae9980356d30e5af/users/b7202c1005f54d89bcc5fa87da62083d/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:13.533840 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bf3518-b6ef-450c-8230-e2bde40ed7c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=985012e7b45949f5ae9980356d30e5af, user_id=b7202c1005f54d89bcc5fa87da62083d, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:13.536459 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bf3518-b6ef-450c-8230-e2bde40ed7c1 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:13.553983 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1312/2639] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:16:13 2022] PUT /identity/v3/projects/985012e7b45949f5ae9980356d30e5af/users/b7202c1005f54d89bcc5fa87da62083d/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:16:13.573444 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10d7df-1f77-407a-9d99-98f69a475faf None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:13.573943 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10d7df-1f77-407a-9d99-98f69a475faf None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:13.574336 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10d7df-1f77-407a-9d99-98f69a475faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:13.610927 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-6c10d7df-1f77-407a-9d99-98f69a475faf None None] MFA Rules not processed for user `b7202c1005f54d89bcc5fa87da62083d`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:13.612777 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-6c10d7df-1f77-407a-9d99-98f69a475faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:13.684842 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1328/2640] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:16:13.697532 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf3518-b6ef-450c-8230-e2bde40ed7c1 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:13.703766 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-02bf3518-b6ef-450c-8230-e2bde40ed7c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:13.722669 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:13.809669 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf3518-b6ef-450c-8230-e2bde40ed7c1 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:13.816274 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81206fc-2fe1-4300-a09a-c63a89a246b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:13.816931 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81206fc-2fe1-4300-a09a-c63a89a246b3 service nova] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:13.817213 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81206fc-2fe1-4300-a09a-c63a89a246b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:13.817482 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81206fc-2fe1-4300-a09a-c63a89a246b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:13.826253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81206fc-2fe1-4300-a09a-c63a89a246b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:13.828774 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81206fc-2fe1-4300-a09a-c63a89a246b3 service nova] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:13.839379 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1313/2641] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:16:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:16:14.299647 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10d7df-1f77-407a-9d99-98f69a475faf None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:14.302245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-6c10d7df-1f77-407a-9d99-98f69a475faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:14.379814 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10d7df-1f77-407a-9d99-98f69a475faf None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:14.383764 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ac32e-42c2-4d3f-b02c-367e16b4514a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:14.384383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ac32e-42c2-4d3f-b02c-367e16b4514a service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:14.384686 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ac32e-42c2-4d3f-b02c-367e16b4514a service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:14.384960 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ac32e-42c2-4d3f-b02c-367e16b4514a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:14.386814 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e04ac32e-42c2-4d3f-b02c-367e16b4514a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:14.436776 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:14.459318 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ac32e-42c2-4d3f-b02c-367e16b4514a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:14.460727 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ac32e-42c2-4d3f-b02c-367e16b4514a service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:14.472303 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1329/2642] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:16:14 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:16:14.529273 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81206fc-2fe1-4300-a09a-c63a89a246b3 service nova] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:14.530752 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-f81206fc-2fe1-4300-a09a-c63a89a246b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:14.627351 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81206fc-2fe1-4300-a09a-c63a89a246b3 service nova] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:14.632244 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b846e-8a48-4b94-bf4e-c5476d5251d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:14.632962 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b846e-8a48-4b94-bf4e-c5476d5251d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:14.633242 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b846e-8a48-4b94-bf4e-c5476d5251d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:14.633504 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b846e-8a48-4b94-bf4e-c5476d5251d5 admin admin] PATH_INFO: `/v3/users/d2e71ee95f0c4c098a27d24ad6ebaf11` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:14.634678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b846e-8a48-4b94-bf4e-c5476d5251d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2e71ee95f0c4c098a27d24ad6ebaf11)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:14.636602 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b846e-8a48-4b94-bf4e-c5476d5251d5 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:14.695178 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-1e6b846e-8a48-4b94-bf4e-c5476d5251d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2e71ee95f0c4c098a27d24ad6ebaf11'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:14.706403 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-1e6b846e-8a48-4b94-bf4e-c5476d5251d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2e71ee95f0c4c098a27d24ad6ebaf11'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:14.722456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-1e6b846e-8a48-4b94-bf4e-c5476d5251d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2e71ee95f0c4c098a27d24ad6ebaf11'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:14.723115 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-1e6b846e-8a48-4b94-bf4e-c5476d5251d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2e71ee95f0c4c098a27d24ad6ebaf11'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:14.733637 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-1e6b846e-8a48-4b94-bf4e-c5476d5251d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2e71ee95f0c4c098a27d24ad6ebaf11'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:14.741281 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1314/2643] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:16:14 2022] DELETE /identity/v3/users/d2e71ee95f0c4c098a27d24ad6ebaf11 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:16:14.746869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686bd7bd-f3eb-4323-a4e3-6a668b75cafc None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:14.747468 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686bd7bd-f3eb-4323-a4e3-6a668b75cafc None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:14.748019 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686bd7bd-f3eb-4323-a4e3-6a668b75cafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:14.773604 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-686bd7bd-f3eb-4323-a4e3-6a668b75cafc None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:14.775791 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-686bd7bd-f3eb-4323-a4e3-6a668b75cafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:14.877253 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1330/2644] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:16:14.890982 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b846e-8a48-4b94-bf4e-c5476d5251d5 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:14.893244 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-1e6b846e-8a48-4b94-bf4e-c5476d5251d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:14.940517 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:14.998023 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b846e-8a48-4b94-bf4e-c5476d5251d5 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:15.003215 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f626a0-ad63-49c3-8ddc-0a49383b6b15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:15.003858 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f626a0-ad63-49c3-8ddc-0a49383b6b15 service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:15.004129 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f626a0-ad63-49c3-8ddc-0a49383b6b15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:15.004496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f626a0-ad63-49c3-8ddc-0a49383b6b15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:15.014844 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f626a0-ad63-49c3-8ddc-0a49383b6b15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:15.017067 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f626a0-ad63-49c3-8ddc-0a49383b6b15 service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:15.028743 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1315/2645] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:16:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:16:15.452204 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686bd7bd-f3eb-4323-a4e3-6a668b75cafc None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:15.454728 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-686bd7bd-f3eb-4323-a4e3-6a668b75cafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:15.472322 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686bd7bd-f3eb-4323-a4e3-6a668b75cafc None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:15.480745 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc4b28a-a3f2-4c3e-911b-2471b1254b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:15.482082 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4b28a-a3f2-4c3e-911b-2471b1254b66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:15.482568 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4b28a-a3f2-4c3e-911b-2471b1254b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:15.483051 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4b28a-a3f2-4c3e-911b-2471b1254b66 admin admin] PATH_INFO: `/v3/projects/86983b89b4404c8e82470dc5227d541b` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:15.485731 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc4b28a-a3f2-4c3e-911b-2471b1254b66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86983b89b4404c8e82470dc5227d541b)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:15.489866 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc4b28a-a3f2-4c3e-911b-2471b1254b66 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:15.571791 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-edc4b28a-a3f2-4c3e-911b-2471b1254b66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86983b89b4404c8e82470dc5227d541b', 'request_id': 'req-edc4b28a-a3f2-4c3e-911b-2471b1254b66'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:15.580173 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-edc4b28a-a3f2-4c3e-911b-2471b1254b66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86983b89b4404c8e82470dc5227d541b', 'request_id': 'req-edc4b28a-a3f2-4c3e-911b-2471b1254b66'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:15.590421 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-edc4b28a-a3f2-4c3e-911b-2471b1254b66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86983b89b4404c8e82470dc5227d541b', 'request_id': 'req-edc4b28a-a3f2-4c3e-911b-2471b1254b66'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:15.612848 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-edc4b28a-a3f2-4c3e-911b-2471b1254b66 admin admin] The token cache is being invalidate because project 86983b89b4404c8e82470dc5227d541b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:16:15.613641 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-edc4b28a-a3f2-4c3e-911b-2471b1254b66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:15.615470 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-edc4b28a-a3f2-4c3e-911b-2471b1254b66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:15.621602 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1331/2646] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:16:15 2022] DELETE /identity/v3/projects/86983b89b4404c8e82470dc5227d541b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:16:15.626075 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f626a0-ad63-49c3-8ddc-0a49383b6b15 service neutron] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:15.646830 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f626a0-ad63-49c3-8ddc-0a49383b6b15 service neutron] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:15.745870 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d95c98-3aff-40c0-a8d4-c599457630cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:15.746860 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d95c98-3aff-40c0-a8d4-c599457630cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:15.747387 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d95c98-3aff-40c0-a8d4-c599457630cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:15.747895 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d95c98-3aff-40c0-a8d4-c599457630cc admin admin] PATH_INFO: `/v3/users/504b7fe3acbd42d3ac760aae3fba61a9` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:15.751025 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d95c98-3aff-40c0-a8d4-c599457630cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=504b7fe3acbd42d3ac760aae3fba61a9)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:15.755277 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d95c98-3aff-40c0-a8d4-c599457630cc admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:15.823786 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-a8d95c98-3aff-40c0-a8d4-c599457630cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '504b7fe3acbd42d3ac760aae3fba61a9'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:15.833026 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-a8d95c98-3aff-40c0-a8d4-c599457630cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '504b7fe3acbd42d3ac760aae3fba61a9'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:15.848399 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-a8d95c98-3aff-40c0-a8d4-c599457630cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '504b7fe3acbd42d3ac760aae3fba61a9'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:15.849359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-a8d95c98-3aff-40c0-a8d4-c599457630cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '504b7fe3acbd42d3ac760aae3fba61a9'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:15.862480 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-a8d95c98-3aff-40c0-a8d4-c599457630cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '504b7fe3acbd42d3ac760aae3fba61a9'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:15.874473 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1316/2647] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:16:15 2022] DELETE /identity/v3/users/504b7fe3acbd42d3ac760aae3fba61a9 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:16:16.246784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc4b28a-a3f2-4c3e-911b-2471b1254b66 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:16.248612 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-edc4b28a-a3f2-4c3e-911b-2471b1254b66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:16.270806 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc4b28a-a3f2-4c3e-911b-2471b1254b66 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:16.362454 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966b0c7-cdb1-4139-9338-ec9a4b33536c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:16.363439 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966b0c7-cdb1-4139-9338-ec9a4b33536c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:16.363721 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966b0c7-cdb1-4139-9338-ec9a4b33536c admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:16.364125 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966b0c7-cdb1-4139-9338-ec9a4b33536c admin admin] PATH_INFO: `/v3/projects/775b55c0a2eb4fb6b0a81d94e500e939` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:16.365913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d966b0c7-cdb1-4139-9338-ec9a4b33536c admin admin] RBAC: Authorizing `identity:delete_project(project_id=775b55c0a2eb4fb6b0a81d94e500e939)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:16.368698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d966b0c7-cdb1-4139-9338-ec9a4b33536c admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:16.378264 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:16.440279 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-d966b0c7-cdb1-4139-9338-ec9a4b33536c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '775b55c0a2eb4fb6b0a81d94e500e939', 'request_id': 'req-d966b0c7-cdb1-4139-9338-ec9a4b33536c'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:16.448646 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-d966b0c7-cdb1-4139-9338-ec9a4b33536c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '775b55c0a2eb4fb6b0a81d94e500e939', 'request_id': 'req-d966b0c7-cdb1-4139-9338-ec9a4b33536c'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:16.459590 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-d966b0c7-cdb1-4139-9338-ec9a4b33536c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '775b55c0a2eb4fb6b0a81d94e500e939', 'request_id': 'req-d966b0c7-cdb1-4139-9338-ec9a4b33536c'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:16.477965 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-d966b0c7-cdb1-4139-9338-ec9a4b33536c admin admin] The token cache is being invalidate because project 775b55c0a2eb4fb6b0a81d94e500e939 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:16:16.477965 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-d966b0c7-cdb1-4139-9338-ec9a4b33536c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:16.478459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-d966b0c7-cdb1-4139-9338-ec9a4b33536c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:16.480997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1332/2648] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:16:16 2022] DELETE /identity/v3/projects/775b55c0a2eb4fb6b0a81d94e500e939 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:16:16.489010 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f34e5-b653-4acd-b9cc-1b7774e52b5b None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:16.490065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f34e5-b653-4acd-b9cc-1b7774e52b5b None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:16.490065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f34e5-b653-4acd-b9cc-1b7774e52b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:16.527761 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-a18f34e5-b653-4acd-b9cc-1b7774e52b5b None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:16.530292 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-a18f34e5-b653-4acd-b9cc-1b7774e52b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:16.577640 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966b0c7-cdb1-4139-9338-ec9a4b33536c admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:16.579553 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-d966b0c7-cdb1-4139-9338-ec9a4b33536c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:16.600497 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:16.640534 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1317/2649] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:16:16.652511 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f2d0e-2d04-4676-a506-15016d4b3ccc None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:16.653069 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f2d0e-2d04-4676-a506-15016d4b3ccc None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:16.653574 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f2d0e-2d04-4676-a506-15016d4b3ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:16.678868 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-de6f2d0e-2d04-4676-a506-15016d4b3ccc None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:16.680086 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-de6f2d0e-2d04-4676-a506-15016d4b3ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:16.689658 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1318/2650] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:16:16.694013 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f2d0e-2d04-4676-a506-15016d4b3ccc None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:16.703725 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f2d0e-2d04-4676-a506-15016d4b3ccc None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:16.708994 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd898799-c5aa-40b4-bfc4-70bb73bd3f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:16.709522 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd898799-c5aa-40b4-bfc4-70bb73bd3f1f admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:16.709822 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd898799-c5aa-40b4-bfc4-70bb73bd3f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:16.710121 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd898799-c5aa-40b4-bfc4-70bb73bd3f1f admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:16.710745 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd898799-c5aa-40b4-bfc4-70bb73bd3f1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:16:16.711051 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd898799-c5aa-40b4-bfc4-70bb73bd3f1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:16.712667 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd898799-c5aa-40b4-bfc4-70bb73bd3f1f admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:16.732635 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1319/2651] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:16:16.733534 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd898799-c5aa-40b4-bfc4-70bb73bd3f1f admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:16.741777 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966b0c7-cdb1-4139-9338-ec9a4b33536c admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:16.744713 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd898799-c5aa-40b4-bfc4-70bb73bd3f1f admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:16.751077 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e20b9-a817-40cb-9ad5-4b31c9141f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:16.751803 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e20b9-a817-40cb-9ad5-4b31c9141f9e admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:16.752152 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e20b9-a817-40cb-9ad5-4b31c9141f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:16.752479 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e20b9-a817-40cb-9ad5-4b31c9141f9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:16.753404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1e20b9-a817-40cb-9ad5-4b31c9141f9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:16.755580 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1e20b9-a817-40cb-9ad5-4b31c9141f9e admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:16.780176 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70274d44-774a-4757-b8f2-ef02816790c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:16.781965 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70274d44-774a-4757-b8f2-ef02816790c2 service nova] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:16.782457 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70274d44-774a-4757-b8f2-ef02816790c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:16.783006 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70274d44-774a-4757-b8f2-ef02816790c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:16.786556 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-70274d44-774a-4757-b8f2-ef02816790c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:16.802368 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1320/2652] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:16:16 2022] POST /identity/v3/projects => generated 373 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:16:16.803416 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e20b9-a817-40cb-9ad5-4b31c9141f9e admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:16.807607 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70274d44-774a-4757-b8f2-ef02816790c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:16.810131 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70274d44-774a-4757-b8f2-ef02816790c2 service nova] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:16.833933 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1333/2653] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:16:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:16:16.838634 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70274d44-774a-4757-b8f2-ef02816790c2 service nova] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:16.840229 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-70274d44-774a-4757-b8f2-ef02816790c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:16.856090 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70274d44-774a-4757-b8f2-ef02816790c2 service nova] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:16.893680 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e20b9-a817-40cb-9ad5-4b31c9141f9e admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:16.900304 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc128a5-b161-45c1-8e29-3823f221e827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:16.901136 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc128a5-b161-45c1-8e29-3823f221e827 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:16.901432 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc128a5-b161-45c1-8e29-3823f221e827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:16.901809 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc128a5-b161-45c1-8e29-3823f221e827 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:16.902561 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc128a5-b161-45c1-8e29-3823f221e827 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:16.904758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc128a5-b161-45c1-8e29-3823f221e827 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:16.936861 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1321/2654] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:16:16 2022] POST /identity/v3/users => generated 287 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:16:16.937328 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded1cb8-dfb4-42b6-af17-412f269bc27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:16.938482 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded1cb8-dfb4-42b6-af17-412f269bc27f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:16.938889 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc128a5-b161-45c1-8e29-3823f221e827 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:16.939149 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded1cb8-dfb4-42b6-af17-412f269bc27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:16.939660 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded1cb8-dfb4-42b6-af17-412f269bc27f admin admin] PATH_INFO: `/v3/users/b7202c1005f54d89bcc5fa87da62083d` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:16.942613 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded1cb8-dfb4-42b6-af17-412f269bc27f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7202c1005f54d89bcc5fa87da62083d)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:16.948044 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded1cb8-dfb4-42b6-af17-412f269bc27f admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:16.953762 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc128a5-b161-45c1-8e29-3823f221e827 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:16.961195 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d7fa36-357e-418d-87b8-fe6aaf6c4850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:16.962201 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7fa36-357e-418d-87b8-fe6aaf6c4850 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:16.962663 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7fa36-357e-418d-87b8-fe6aaf6c4850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:16.962944 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7fa36-357e-418d-87b8-fe6aaf6c4850 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:16.964078 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d7fa36-357e-418d-87b8-fe6aaf6c4850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:16.966452 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d7fa36-357e-418d-87b8-fe6aaf6c4850 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:16.982733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1322/2655] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:16:16.986092 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:16.989946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d7fa36-357e-418d-87b8-fe6aaf6c4850 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:17.002875 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d7fa36-357e-418d-87b8-fe6aaf6c4850 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:17.010975 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c6820-5947-42d7-9e0b-1559b4ba8887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:17.011885 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c6820-5947-42d7-9e0b-1559b4ba8887 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:17.012217 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c6820-5947-42d7-9e0b-1559b4ba8887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:17.012608 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c6820-5947-42d7-9e0b-1559b4ba8887 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:17.013365 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c6820-5947-42d7-9e0b-1559b4ba8887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:17.016269 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c6820-5947-42d7-9e0b-1559b4ba8887 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:17.035296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1323/2656] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:16:17.036246 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c6820-5947-42d7-9e0b-1559b4ba8887 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:17.038815 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-3ded1cb8-dfb4-42b6-af17-412f269bc27f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7202c1005f54d89bcc5fa87da62083d'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:17.049528 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-3ded1cb8-dfb4-42b6-af17-412f269bc27f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7202c1005f54d89bcc5fa87da62083d'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:17.059028 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-3ded1cb8-dfb4-42b6-af17-412f269bc27f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7202c1005f54d89bcc5fa87da62083d'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:17.059334 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c6820-5947-42d7-9e0b-1559b4ba8887 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:17.067065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b9174-31ad-471e-88e5-f1130a2c70b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:17.068381 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b9174-31ad-471e-88e5-f1130a2c70b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:17.069065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b9174-31ad-471e-88e5-f1130a2c70b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:17.069548 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b9174-31ad-471e-88e5-f1130a2c70b9 admin admin] PATH_INFO: `/v3/projects/0ca5a95643be4621bb705111b135c214/users/98f007e401394276881d506bb2fccec5/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:17.074217 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-3ded1cb8-dfb4-42b6-af17-412f269bc27f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7202c1005f54d89bcc5fa87da62083d'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:17.075262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-3ded1cb8-dfb4-42b6-af17-412f269bc27f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7202c1005f54d89bcc5fa87da62083d'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:17.088937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1334/2657] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:16:16 2022] DELETE /identity/v3/users/b7202c1005f54d89bcc5fa87da62083d => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:16:17.091541 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7acb68a-cf32-4ecb-acc3-cd7862b67026 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:17.092105 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7acb68a-cf32-4ecb-acc3-cd7862b67026 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:17.092629 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7acb68a-cf32-4ecb-acc3-cd7862b67026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:17.099971 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1b9174-31ad-471e-88e5-f1130a2c70b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ca5a95643be4621bb705111b135c214, user_id=98f007e401394276881d506bb2fccec5, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:17.105996 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1b9174-31ad-471e-88e5-f1130a2c70b9 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:17.126991 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-f7acb68a-cf32-4ecb-acc3-cd7862b67026 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:17.128251 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-f7acb68a-cf32-4ecb-acc3-cd7862b67026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:17.138017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1324/2658] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:16:17 2022] PUT /identity/v3/projects/0ca5a95643be4621bb705111b135c214/users/98f007e401394276881d506bb2fccec5/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:16:17.151650 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10359808-953b-47a8-bd1b-f69428b35d3a None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:17.152471 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10359808-953b-47a8-bd1b-f69428b35d3a None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:17.153126 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10359808-953b-47a8-bd1b-f69428b35d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:17.208892 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-10359808-953b-47a8-bd1b-f69428b35d3a None None] MFA Rules not processed for user `98f007e401394276881d506bb2fccec5`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:17.211017 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-10359808-953b-47a8-bd1b-f69428b35d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:17.252032 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1335/2659] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:16:17.266375 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:17.268366 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7acb68a-cf32-4ecb-acc3-cd7862b67026 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:17.271283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-f7acb68a-cf32-4ecb-acc3-cd7862b67026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:17.327696 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1325/2660] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:16:17.345473 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10359808-953b-47a8-bd1b-f69428b35d3a None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:17.347310 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-10359808-953b-47a8-bd1b-f69428b35d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:17.384906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7acb68a-cf32-4ecb-acc3-cd7862b67026 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:17.389670 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7748f-e60b-4528-b03b-df5d22a12228 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:17.390342 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7748f-e60b-4528-b03b-df5d22a12228 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:17.390679 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7748f-e60b-4528-b03b-df5d22a12228 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:17.391179 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7748f-e60b-4528-b03b-df5d22a12228 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:17.418896 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca7748f-e60b-4528-b03b-df5d22a12228 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:17.422678 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca7748f-e60b-4528-b03b-df5d22a12228 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:17.436529 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1336/2661] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:16:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:16:17.442578 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10359808-953b-47a8-bd1b-f69428b35d3a None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:17.449636 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c8a5d-e86e-42d4-953a-2aff0c5c51f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:17.451040 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c8a5d-e86e-42d4-953a-2aff0c5c51f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:17.451564 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c8a5d-e86e-42d4-953a-2aff0c5c51f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:17.451950 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c8a5d-e86e-42d4-953a-2aff0c5c51f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:17.471640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492c8a5d-e86e-42d4-953a-2aff0c5c51f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:17.475610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492c8a5d-e86e-42d4-953a-2aff0c5c51f5 service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:17.497428 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1326/2662] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:16:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:16:17.547327 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7748f-e60b-4528-b03b-df5d22a12228 service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:17.549655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-bca7748f-e60b-4528-b03b-df5d22a12228 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:17.575814 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7748f-e60b-4528-b03b-df5d22a12228 service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:17.584591 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3307b3-470f-4b67-92b4-f5e06b3ff3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:17.585937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3307b3-470f-4b67-92b4-f5e06b3ff3e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:17.586326 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3307b3-470f-4b67-92b4-f5e06b3ff3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:17.586773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3307b3-470f-4b67-92b4-f5e06b3ff3e4 admin admin] PATH_INFO: `/v3/users/98f007e401394276881d506bb2fccec5` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:17.588378 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3307b3-470f-4b67-92b4-f5e06b3ff3e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98f007e401394276881d506bb2fccec5)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:17.590536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3307b3-470f-4b67-92b4-f5e06b3ff3e4 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:17.660189 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:17.678875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-6f3307b3-470f-4b67-92b4-f5e06b3ff3e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98f007e401394276881d506bb2fccec5'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:17.691285 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-6f3307b3-470f-4b67-92b4-f5e06b3ff3e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98f007e401394276881d506bb2fccec5'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:17.702217 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-6f3307b3-470f-4b67-92b4-f5e06b3ff3e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98f007e401394276881d506bb2fccec5'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:17.725443 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-6f3307b3-470f-4b67-92b4-f5e06b3ff3e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98f007e401394276881d506bb2fccec5'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:17.726701 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-6f3307b3-470f-4b67-92b4-f5e06b3ff3e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98f007e401394276881d506bb2fccec5'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:17.740044 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1337/2663] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:16:17 2022] DELETE /identity/v3/users/98f007e401394276881d506bb2fccec5 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:16:17.741479 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9aba03-1143-45d2-a821-b5b2c01e2372 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:17.742226 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9aba03-1143-45d2-a821-b5b2c01e2372 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:17.742226 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9aba03-1143-45d2-a821-b5b2c01e2372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:17.777242 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-eb9aba03-1143-45d2-a821-b5b2c01e2372 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:17.778599 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-eb9aba03-1143-45d2-a821-b5b2c01e2372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:17.907418 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1327/2664] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:16:17.916021 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3307b3-470f-4b67-92b4-f5e06b3ff3e4 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:17.917712 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-6f3307b3-470f-4b67-92b4-f5e06b3ff3e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:17.945924 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9aba03-1143-45d2-a821-b5b2c01e2372 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:17.950546 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-eb9aba03-1143-45d2-a821-b5b2c01e2372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:17.955968 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3307b3-470f-4b67-92b4-f5e06b3ff3e4 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:17.962740 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77129b91-fd3f-4524-8741-b8cfef697385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:17.963609 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77129b91-fd3f-4524-8741-b8cfef697385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:17.963609 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77129b91-fd3f-4524-8741-b8cfef697385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:17.966131 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77129b91-fd3f-4524-8741-b8cfef697385 admin admin] PATH_INFO: `/v3/projects/985012e7b45949f5ae9980356d30e5af` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:17.966131 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77129b91-fd3f-4524-8741-b8cfef697385 admin admin] RBAC: Authorizing `identity:delete_project(project_id=985012e7b45949f5ae9980356d30e5af)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:17.968166 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77129b91-fd3f-4524-8741-b8cfef697385 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:18.052569 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-77129b91-fd3f-4524-8741-b8cfef697385 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '985012e7b45949f5ae9980356d30e5af', 'request_id': 'req-77129b91-fd3f-4524-8741-b8cfef697385'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:18.052920 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9aba03-1143-45d2-a821-b5b2c01e2372 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:18.063970 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-77129b91-fd3f-4524-8741-b8cfef697385 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '985012e7b45949f5ae9980356d30e5af', 'request_id': 'req-77129b91-fd3f-4524-8741-b8cfef697385'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:18.076156 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-77129b91-fd3f-4524-8741-b8cfef697385 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '985012e7b45949f5ae9980356d30e5af', 'request_id': 'req-77129b91-fd3f-4524-8741-b8cfef697385'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:18.099403 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-77129b91-fd3f-4524-8741-b8cfef697385 admin admin] The token cache is being invalidate because project 985012e7b45949f5ae9980356d30e5af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:16:18.099920 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-77129b91-fd3f-4524-8741-b8cfef697385 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:18.100965 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-77129b91-fd3f-4524-8741-b8cfef697385 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:18.103096 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1338/2665] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:16:17 2022] DELETE /identity/v3/projects/985012e7b45949f5ae9980356d30e5af => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:16:18.108464 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d3d8e-b879-4bb5-9bea-3f3d209ae1c1 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:18.108840 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d3d8e-b879-4bb5-9bea-3f3d209ae1c1 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:18.109277 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d3d8e-b879-4bb5-9bea-3f3d209ae1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:18.112866 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:18.139659 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-699d3d8e-b879-4bb5-9bea-3f3d209ae1c1 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:18.142425 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-699d3d8e-b879-4bb5-9bea-3f3d209ae1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:18.151786 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3810b9ff-8d85-4a0b-b348-9c061d57ca9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:18.152817 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810b9ff-8d85-4a0b-b348-9c061d57ca9b service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:18.153365 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810b9ff-8d85-4a0b-b348-9c061d57ca9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:18.153787 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810b9ff-8d85-4a0b-b348-9c061d57ca9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:18.161863 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-3810b9ff-8d85-4a0b-b348-9c061d57ca9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:18.238210 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1339/2666] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:16:18.247142 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658e702-1df9-4273-991f-c134db7c1325 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:18.247584 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658e702-1df9-4273-991f-c134db7c1325 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:18.248227 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658e702-1df9-4273-991f-c134db7c1325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:18.264223 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3810b9ff-8d85-4a0b-b348-9c061d57ca9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:18.268044 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3810b9ff-8d85-4a0b-b348-9c061d57ca9b service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:18.272436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-3658e702-1df9-4273-991f-c134db7c1325 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:18.274257 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-3658e702-1df9-4273-991f-c134db7c1325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:18.290087 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1340/2667] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:16:18.290323 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1328/2668] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:16:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:16:18.292159 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658e702-1df9-4273-991f-c134db7c1325 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:18.303824 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658e702-1df9-4273-991f-c134db7c1325 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:18.310069 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f6467b-79e2-407f-bf0d-d36fb339e21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:18.310858 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6467b-79e2-407f-bf0d-d36fb339e21b admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:18.311149 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6467b-79e2-407f-bf0d-d36fb339e21b admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:18.311426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6467b-79e2-407f-bf0d-d36fb339e21b admin admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:18.312872 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f6467b-79e2-407f-bf0d-d36fb339e21b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:16:18.313270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f6467b-79e2-407f-bf0d-d36fb339e21b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:18.316238 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f6467b-79e2-407f-bf0d-d36fb339e21b admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:18.338061 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1341/2669] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:16:18.345917 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3810b9ff-8d85-4a0b-b348-9c061d57ca9b service neutron] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:18.366600 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3810b9ff-8d85-4a0b-b348-9c061d57ca9b service neutron] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:18.373210 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc168378-b674-4698-90d8-10e7166a1a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:18.373858 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc168378-b674-4698-90d8-10e7166a1a5b admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:18.374210 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc168378-b674-4698-90d8-10e7166a1a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:18.374404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc168378-b674-4698-90d8-10e7166a1a5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:18.375188 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc168378-b674-4698-90d8-10e7166a1a5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:18.377306 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc168378-b674-4698-90d8-10e7166a1a5b admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:18.418008 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1329/2670] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:16:18 2022] POST /identity/v3/projects => generated 387 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:16:18.419966 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f6467b-79e2-407f-bf0d-d36fb339e21b admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:18.432682 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f6467b-79e2-407f-bf0d-d36fb339e21b admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:18.462952 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:18.563637 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e7523-ec2a-4ada-a4a1-f5a7784d32ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:18.564934 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e7523-ec2a-4ada-a4a1-f5a7784d32ff admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:18.565375 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e7523-ec2a-4ada-a4a1-f5a7784d32ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:18.565808 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e7523-ec2a-4ada-a4a1-f5a7784d32ff admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:18.566646 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429e7523-ec2a-4ada-a4a1-f5a7784d32ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:18.569041 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429e7523-ec2a-4ada-a4a1-f5a7784d32ff admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:18.598324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1342/2671] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:16:18 2022] POST /identity/v3/users => generated 294 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:16:18.603142 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc168378-b674-4698-90d8-10e7166a1a5b admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:18.616059 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc168378-b674-4698-90d8-10e7166a1a5b admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:18.624144 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f7c80-5dc3-4f10-b992-72cd3e7fde45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:18.625569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f7c80-5dc3-4f10-b992-72cd3e7fde45 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:18.626108 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f7c80-5dc3-4f10-b992-72cd3e7fde45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:18.626579 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f7c80-5dc3-4f10-b992-72cd3e7fde45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:18.627787 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f7c80-5dc3-4f10-b992-72cd3e7fde45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:18.629949 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f7c80-5dc3-4f10-b992-72cd3e7fde45 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:18.643225 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1330/2672] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:16:18.649799 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e7523-ec2a-4ada-a4a1-f5a7784d32ff admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:18.659196 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f7c80-5dc3-4f10-b992-72cd3e7fde45 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:18.662257 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-1f7f7c80-5dc3-4f10-b992-72cd3e7fde45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:18.664691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e7523-ec2a-4ada-a4a1-f5a7784d32ff admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:18.669793 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af77242-cd15-4b52-9514-aff5bf6bc0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:18.670481 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af77242-cd15-4b52-9514-aff5bf6bc0f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:18.670771 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af77242-cd15-4b52-9514-aff5bf6bc0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:18.671049 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af77242-cd15-4b52-9514-aff5bf6bc0f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:18.672548 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af77242-cd15-4b52-9514-aff5bf6bc0f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:18.674100 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af77242-cd15-4b52-9514-aff5bf6bc0f3 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:18.686942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1343/2673] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:16:18.691772 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f7c80-5dc3-4f10-b992-72cd3e7fde45 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:18.695104 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af77242-cd15-4b52-9514-aff5bf6bc0f3 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:18.699674 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84def940-7de7-4471-8398-999825a5c9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:18.700775 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84def940-7de7-4471-8398-999825a5c9d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:18.701261 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84def940-7de7-4471-8398-999825a5c9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:18.701733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84def940-7de7-4471-8398-999825a5c9d5 admin admin] PATH_INFO: `/v3/projects/0ca5a95643be4621bb705111b135c214` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:18.704471 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84def940-7de7-4471-8398-999825a5c9d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ca5a95643be4621bb705111b135c214)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:18.708219 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84def940-7de7-4471-8398-999825a5c9d5 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:18.712515 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af77242-cd15-4b52-9514-aff5bf6bc0f3 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:18.718391 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c34c97a-fd2d-441a-8a8b-6cfb96a0544f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:18.719289 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c34c97a-fd2d-441a-8a8b-6cfb96a0544f admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:18.719676 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c34c97a-fd2d-441a-8a8b-6cfb96a0544f admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:18.720040 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c34c97a-fd2d-441a-8a8b-6cfb96a0544f admin admin] PATH_INFO: `/v3/projects/e120a75dce744ca78c4f36023901c711/users/8e57e95333294a519774bd52f1bc04c5/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:18.737111 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c34c97a-fd2d-441a-8a8b-6cfb96a0544f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e120a75dce744ca78c4f36023901c711, user_id=8e57e95333294a519774bd52f1bc04c5, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:18.740995 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c34c97a-fd2d-441a-8a8b-6cfb96a0544f admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:18.756520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1344/2674] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:16:18 2022] PUT /identity/v3/projects/e120a75dce744ca78c4f36023901c711/users/8e57e95333294a519774bd52f1bc04c5/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:16:18.767853 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189677c4-b1f6-474b-82e8-6c0a482fb4e1 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:18.768170 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189677c4-b1f6-474b-82e8-6c0a482fb4e1 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:18.768371 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189677c4-b1f6-474b-82e8-6c0a482fb4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:18.780773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-84def940-7de7-4471-8398-999825a5c9d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ca5a95643be4621bb705111b135c214', 'request_id': 'req-84def940-7de7-4471-8398-999825a5c9d5'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:18.792139 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-84def940-7de7-4471-8398-999825a5c9d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ca5a95643be4621bb705111b135c214', 'request_id': 'req-84def940-7de7-4471-8398-999825a5c9d5'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:18.799547 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-189677c4-b1f6-474b-82e8-6c0a482fb4e1 None None] MFA Rules not processed for user `8e57e95333294a519774bd52f1bc04c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:18.800905 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-189677c4-b1f6-474b-82e8-6c0a482fb4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:18.803156 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-84def940-7de7-4471-8398-999825a5c9d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ca5a95643be4621bb705111b135c214', 'request_id': 'req-84def940-7de7-4471-8398-999825a5c9d5'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:18.822946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-84def940-7de7-4471-8398-999825a5c9d5 admin admin] The token cache is being invalidate because project 0ca5a95643be4621bb705111b135c214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:16:18.823408 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-84def940-7de7-4471-8398-999825a5c9d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:18.824284 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-84def940-7de7-4471-8398-999825a5c9d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:18.826293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1331/2675] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:16:18 2022] DELETE /identity/v3/projects/0ca5a95643be4621bb705111b135c214 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:16:18.832928 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad60e8-3d41-42bb-b752-65b7c1beed3e None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:18.833268 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad60e8-3d41-42bb-b752-65b7c1beed3e None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:18.833696 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad60e8-3d41-42bb-b752-65b7c1beed3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:18.855517 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-58ad60e8-3d41-42bb-b752-65b7c1beed3e None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:18.857445 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-58ad60e8-3d41-42bb-b752-65b7c1beed3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:18.889671 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1345/2676] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:16:18.895598 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189677c4-b1f6-474b-82e8-6c0a482fb4e1 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:18.896158 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:18.898458 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-189677c4-b1f6-474b-82e8-6c0a482fb4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:18.938998 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1332/2677] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:16:18.949509 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e60496-2c97-4992-accf-d92427aec931 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:18.950135 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e60496-2c97-4992-accf-d92427aec931 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:18.950583 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e60496-2c97-4992-accf-d92427aec931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:18.972261 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-c6e60496-2c97-4992-accf-d92427aec931 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:18.973484 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-c6e60496-2c97-4992-accf-d92427aec931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:18.978348 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189677c4-b1f6-474b-82e8-6c0a482fb4e1 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:18.981442 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1333/2678] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:16:18.985714 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e60496-2c97-4992-accf-d92427aec931 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:18.987567 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7898b618-c7c5-4e48-b867-acdc1eda1fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:18.988286 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898b618-c7c5-4e48-b867-acdc1eda1fe5 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:18.988755 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898b618-c7c5-4e48-b867-acdc1eda1fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:18.989127 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898b618-c7c5-4e48-b867-acdc1eda1fe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:18.989887 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7898b618-c7c5-4e48-b867-acdc1eda1fe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:18.992166 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7898b618-c7c5-4e48-b867-acdc1eda1fe5 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:18.997887 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e60496-2c97-4992-accf-d92427aec931 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:19.003715 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:19.004085 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6fb5d0-d5b0-4565-98e6-c5912e620e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:19.004721 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6fb5d0-d5b0-4565-98e6-c5912e620e13 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:19.005006 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6fb5d0-d5b0-4565-98e6-c5912e620e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:19.005269 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6fb5d0-d5b0-4565-98e6-c5912e620e13 admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:19.005958 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6fb5d0-d5b0-4565-98e6-c5912e620e13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:16:19.006324 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6fb5d0-d5b0-4565-98e6-c5912e620e13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:19.008102 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6fb5d0-d5b0-4565-98e6-c5912e620e13 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:19.022595 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1334/2679] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 20:16:19.027067 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6fb5d0-d5b0-4565-98e6-c5912e620e13 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:19.027783 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1346/2680] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:16:18 2022] POST /identity/v3/projects => generated 389 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:16:19.032628 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7898b618-c7c5-4e48-b867-acdc1eda1fe5 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:19.043335 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6fb5d0-d5b0-4565-98e6-c5912e620e13 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:19.116056 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665130eb-b546-496d-819b-5d3e3a721413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:19.116797 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665130eb-b546-496d-819b-5d3e3a721413 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:19.117080 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665130eb-b546-496d-819b-5d3e3a721413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:19.117355 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665130eb-b546-496d-819b-5d3e3a721413 admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:19.118037 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665130eb-b546-496d-819b-5d3e3a721413 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:19.119938 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665130eb-b546-496d-819b-5d3e3a721413 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:19.122846 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7898b618-c7c5-4e48-b867-acdc1eda1fe5 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:19.129318 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff69e9e-4870-488e-9fe1-7d3daeb172eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:19.129946 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff69e9e-4870-488e-9fe1-7d3daeb172eb admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:19.130225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff69e9e-4870-488e-9fe1-7d3daeb172eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:19.130506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff69e9e-4870-488e-9fe1-7d3daeb172eb admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:19.131180 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff69e9e-4870-488e-9fe1-7d3daeb172eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:19.133151 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff69e9e-4870-488e-9fe1-7d3daeb172eb admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:19.162193 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1347/2681] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:16:19 2022] POST /identity/v3/users => generated 301 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:16:19.163317 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1335/2682] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:16:19 2022] POST /identity/v3/projects => generated 397 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:16:19.167609 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff69e9e-4870-488e-9fe1-7d3daeb172eb admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:19.168922 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665130eb-b546-496d-819b-5d3e3a721413 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:19.181502 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff69e9e-4870-488e-9fe1-7d3daeb172eb admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:19.257973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665130eb-b546-496d-819b-5d3e3a721413 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:19.262952 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467e532-73ef-4f4b-8ee7-f521c4115791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:19.263577 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467e532-73ef-4f4b-8ee7-f521c4115791 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:19.263851 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467e532-73ef-4f4b-8ee7-f521c4115791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:19.264120 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467e532-73ef-4f4b-8ee7-f521c4115791 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:19.265000 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7467e532-73ef-4f4b-8ee7-f521c4115791 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:19.266852 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b9a82-446d-4d6e-9eff-4b558e0f3193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:19.267073 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7467e532-73ef-4f4b-8ee7-f521c4115791 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:19.267711 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b9a82-446d-4d6e-9eff-4b558e0f3193 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:19.268095 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b9a82-446d-4d6e-9eff-4b558e0f3193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:19.268475 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b9a82-446d-4d6e-9eff-4b558e0f3193 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:19.269307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0b9a82-446d-4d6e-9eff-4b558e0f3193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:19.271488 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0b9a82-446d-4d6e-9eff-4b558e0f3193 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:19.284345 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1348/2683] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:16:19.288914 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b9a82-446d-4d6e-9eff-4b558e0f3193 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:19.294157 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1336/2684] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:16:19 2022] POST /identity/v3/users => generated 299 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:16:19.300453 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467e532-73ef-4f4b-8ee7-f521c4115791 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:19.301213 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b9a82-446d-4d6e-9eff-4b558e0f3193 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:19.306296 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a350bb-e917-45c8-97e1-befb4bd9164a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:19.306988 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a350bb-e917-45c8-97e1-befb4bd9164a admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:19.307275 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a350bb-e917-45c8-97e1-befb4bd9164a admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:19.307732 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a350bb-e917-45c8-97e1-befb4bd9164a admin admin] PATH_INFO: `/v3/projects/30cdbf7283374804a84b35faae8089da/users/98a988be948e4d53abbbd673f9c971ae/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:19.316050 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467e532-73ef-4f4b-8ee7-f521c4115791 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:19.321519 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982319b-d051-41db-9396-bad51d4e95c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:19.322595 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982319b-d051-41db-9396-bad51d4e95c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:19.322962 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982319b-d051-41db-9396-bad51d4e95c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:19.323324 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982319b-d051-41db-9396-bad51d4e95c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:19.324493 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982319b-d051-41db-9396-bad51d4e95c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:19.327018 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982319b-d051-41db-9396-bad51d4e95c0 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:19.327674 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a350bb-e917-45c8-97e1-befb4bd9164a admin admin] RBAC: Authorizing `identity:create_grant(project_id=30cdbf7283374804a84b35faae8089da, user_id=98a988be948e4d53abbbd673f9c971ae, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:19.332204 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a350bb-e917-45c8-97e1-befb4bd9164a admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:19.348871 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1337/2685] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:16:19.350049 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982319b-d051-41db-9396-bad51d4e95c0 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:19.358698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1349/2686] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:16:19 2022] PUT /identity/v3/projects/30cdbf7283374804a84b35faae8089da/users/98a988be948e4d53abbbd673f9c971ae/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:16:19.363397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982319b-d051-41db-9396-bad51d4e95c0 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:19.365149 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a350bb-e917-45c8-97e1-befb4bd9164a admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:19.382632 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:19.462635 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548fce1-7b89-42f6-870c-8a8fe4336d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:19.463404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548fce1-7b89-42f6-870c-8a8fe4336d8b admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:19.463789 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548fce1-7b89-42f6-870c-8a8fe4336d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:19.464143 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548fce1-7b89-42f6-870c-8a8fe4336d8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:19.464990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8548fce1-7b89-42f6-870c-8a8fe4336d8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:19.466638 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8548fce1-7b89-42f6-870c-8a8fe4336d8b admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:19.480053 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1338/2687] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:16:19.485397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548fce1-7b89-42f6-870c-8a8fe4336d8b admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:19.496318 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548fce1-7b89-42f6-870c-8a8fe4336d8b admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:19.501563 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d7911-d0af-4912-84b2-10ccc71a7ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:19.502255 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d7911-d0af-4912-84b2-10ccc71a7ed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:19.502697 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d7911-d0af-4912-84b2-10ccc71a7ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:19.502979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d7911-d0af-4912-84b2-10ccc71a7ed7 admin admin] PATH_INFO: `/v3/projects/663b91426b32473ba74764447b10b7cd/users/a5d67da55a9d4541b3cc58ae8c80e6b3/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:19.522342 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d7911-d0af-4912-84b2-10ccc71a7ed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=663b91426b32473ba74764447b10b7cd, user_id=a5d67da55a9d4541b3cc58ae8c80e6b3, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:19.525274 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d7911-d0af-4912-84b2-10ccc71a7ed7 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:19.540674 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1339/2688] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:16:19 2022] PUT /identity/v3/projects/663b91426b32473ba74764447b10b7cd/users/a5d67da55a9d4541b3cc58ae8c80e6b3/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:16:19.552174 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f160c1-0f2a-4efb-a40f-c169433e9be5 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:19.552432 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f160c1-0f2a-4efb-a40f-c169433e9be5 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:19.552813 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f160c1-0f2a-4efb-a40f-c169433e9be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:19.552959 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:19.567264 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a350bb-e917-45c8-97e1-befb4bd9164a admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:19.582969 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-d3f160c1-0f2a-4efb-a40f-c169433e9be5 None None] MFA Rules not processed for user `a5d67da55a9d4541b3cc58ae8c80e6b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:19.584182 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-d3f160c1-0f2a-4efb-a40f-c169433e9be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:19.630337 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1c62b-4746-4e0e-b9af-9d3401387e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:19.631111 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1c62b-4746-4e0e-b9af-9d3401387e85 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:19.631395 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1c62b-4746-4e0e-b9af-9d3401387e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:19.631656 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1c62b-4746-4e0e-b9af-9d3401387e85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:19.632356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed1c62b-4746-4e0e-b9af-9d3401387e85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:19.634009 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed1c62b-4746-4e0e-b9af-9d3401387e85 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:19.646230 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1350/2689] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:16:19.651344 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1c62b-4746-4e0e-b9af-9d3401387e85 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:19.666821 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1c62b-4746-4e0e-b9af-9d3401387e85 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:19.667225 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1340/2690] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:19 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:16:19.676018 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007619cc-2763-48e1-96a5-5a2a3635903c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:19.676710 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007619cc-2763-48e1-96a5-5a2a3635903c admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:19.676990 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007619cc-2763-48e1-96a5-5a2a3635903c admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:19.677269 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007619cc-2763-48e1-96a5-5a2a3635903c admin admin] PATH_INFO: `/v3/projects/30cdbf7283374804a84b35faae8089da/users/98a988be948e4d53abbbd673f9c971ae/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:19.679364 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007619cc-2763-48e1-96a5-5a2a3635903c admin admin] RBAC: Authorizing `identity:create_grant(project_id=30cdbf7283374804a84b35faae8089da, user_id=98a988be948e4d53abbbd673f9c971ae, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:19.681564 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f160c1-0f2a-4efb-a40f-c169433e9be5 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:19.682603 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007619cc-2763-48e1-96a5-5a2a3635903c admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:19.683127 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-d3f160c1-0f2a-4efb-a40f-c169433e9be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:19.695307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1351/2691] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:16:19 2022] PUT /identity/v3/projects/30cdbf7283374804a84b35faae8089da/users/98a988be948e4d53abbbd673f9c971ae/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:16:19.707124 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2559867-b6d8-4983-9c5f-dfd960c5b60c None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:19.707437 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2559867-b6d8-4983-9c5f-dfd960c5b60c None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:19.707654 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2559867-b6d8-4983-9c5f-dfd960c5b60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:19.737065 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-d2559867-b6d8-4983-9c5f-dfd960c5b60c None None] MFA Rules not processed for user `98a988be948e4d53abbbd673f9c971ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:19.738201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-d2559867-b6d8-4983-9c5f-dfd960c5b60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:19.754647 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f160c1-0f2a-4efb-a40f-c169433e9be5 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:19.788438 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:19.812051 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac389e08-f833-464e-9664-0c1c3417b71a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:19.813605 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac389e08-f833-464e-9664-0c1c3417b71a service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:19.813801 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac389e08-f833-464e-9664-0c1c3417b71a service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:19.814077 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac389e08-f833-464e-9664-0c1c3417b71a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:19.825942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1352/2692] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:19 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:16:19.837823 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2559867-b6d8-4983-9c5f-dfd960c5b60c None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:19.839303 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-d2559867-b6d8-4983-9c5f-dfd960c5b60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:19.853462 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2559867-b6d8-4983-9c5f-dfd960c5b60c None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:19.857991 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a0c9d-cf8c-47a7-a5a4-220f6cb9e9b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:19.858709 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a0c9d-cf8c-47a7-a5a4-220f6cb9e9b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:19.859617 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a0c9d-cf8c-47a7-a5a4-220f6cb9e9b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:19.860100 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a0c9d-cf8c-47a7-a5a4-220f6cb9e9b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:19.871307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a0c9d-cf8c-47a7-a5a4-220f6cb9e9b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:19.872839 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a0c9d-cf8c-47a7-a5a4-220f6cb9e9b4 service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:19.884333 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1353/2693] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:16:19 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:16:19.888938 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac389e08-f833-464e-9664-0c1c3417b71a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:19.890460 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac389e08-f833-464e-9664-0c1c3417b71a service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:19.901773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1341/2694] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:16:19 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:16:20.036957 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a0c9d-cf8c-47a7-a5a4-220f6cb9e9b4 service cinder] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:20.097916 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:20.111641 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a0c9d-cf8c-47a7-a5a4-220f6cb9e9b4 service cinder] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:20.117471 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a331b73-a7ef-4696-a3db-2258c18e2859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:20.118169 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a331b73-a7ef-4696-a3db-2258c18e2859 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:20.119338 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a331b73-a7ef-4696-a3db-2258c18e2859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:20.119694 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a331b73-a7ef-4696-a3db-2258c18e2859 admin admin] PATH_INFO: `/v3/users/8e57e95333294a519774bd52f1bc04c5` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:20.121009 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a331b73-a7ef-4696-a3db-2258c18e2859 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e57e95333294a519774bd52f1bc04c5)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:20.123308 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a331b73-a7ef-4696-a3db-2258c18e2859 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:20.181322 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5a331b73-a7ef-4696-a3db-2258c18e2859 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e57e95333294a519774bd52f1bc04c5'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:20.190714 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5a331b73-a7ef-4696-a3db-2258c18e2859 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e57e95333294a519774bd52f1bc04c5'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:20.199524 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5a331b73-a7ef-4696-a3db-2258c18e2859 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e57e95333294a519774bd52f1bc04c5'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:20.215847 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5a331b73-a7ef-4696-a3db-2258c18e2859 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e57e95333294a519774bd52f1bc04c5'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:20.217034 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-5a331b73-a7ef-4696-a3db-2258c18e2859 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e57e95333294a519774bd52f1bc04c5'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:20.226654 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1354/2695] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:16:20 2022] DELETE /identity/v3/users/8e57e95333294a519774bd52f1bc04c5 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:16:20.233336 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc7116-2d83-4a18-bf51-26324ebd768e None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:20.233619 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc7116-2d83-4a18-bf51-26324ebd768e None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:20.233891 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc7116-2d83-4a18-bf51-26324ebd768e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:20.259950 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-fefc7116-2d83-4a18-bf51-26324ebd768e None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:20.261272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-fefc7116-2d83-4a18-bf51-26324ebd768e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:20.332957 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1342/2696] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:16:20.344319 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a331b73-a7ef-4696-a3db-2258c18e2859 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:20.345765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-5a331b73-a7ef-4696-a3db-2258c18e2859 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:20.429729 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a331b73-a7ef-4696-a3db-2258c18e2859 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:20.434811 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6b34c-abd4-4cb4-b9c9-3afdbc496d57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:20.435430 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6b34c-abd4-4cb4-b9c9-3afdbc496d57 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:20.435708 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6b34c-abd4-4cb4-b9c9-3afdbc496d57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:20.435978 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6b34c-abd4-4cb4-b9c9-3afdbc496d57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:20.449103 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6b34c-abd4-4cb4-b9c9-3afdbc496d57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:20.451020 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6b34c-abd4-4cb4-b9c9-3afdbc496d57 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:20.464475 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1355/2697] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:16:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:16:20.756406 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc7116-2d83-4a18-bf51-26324ebd768e None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:20.758426 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-fefc7116-2d83-4a18-bf51-26324ebd768e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:20.785360 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc7116-2d83-4a18-bf51-26324ebd768e None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:20.791425 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2a648-0a5c-4e5f-8cf3-1a28f04b84ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:20.792243 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2a648-0a5c-4e5f-8cf3-1a28f04b84ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:20.792526 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2a648-0a5c-4e5f-8cf3-1a28f04b84ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:20.792846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2a648-0a5c-4e5f-8cf3-1a28f04b84ee admin admin] PATH_INFO: `/v3/projects/e120a75dce744ca78c4f36023901c711` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:20.794042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f2a648-0a5c-4e5f-8cf3-1a28f04b84ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=e120a75dce744ca78c4f36023901c711)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:20.795969 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f2a648-0a5c-4e5f-8cf3-1a28f04b84ee admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:20.856571 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-35f2a648-0a5c-4e5f-8cf3-1a28f04b84ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e120a75dce744ca78c4f36023901c711', 'request_id': 'req-35f2a648-0a5c-4e5f-8cf3-1a28f04b84ee'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:20.871362 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-35f2a648-0a5c-4e5f-8cf3-1a28f04b84ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e120a75dce744ca78c4f36023901c711', 'request_id': 'req-35f2a648-0a5c-4e5f-8cf3-1a28f04b84ee'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:20.880297 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-35f2a648-0a5c-4e5f-8cf3-1a28f04b84ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e120a75dce744ca78c4f36023901c711', 'request_id': 'req-35f2a648-0a5c-4e5f-8cf3-1a28f04b84ee'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:20.891612 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-35f2a648-0a5c-4e5f-8cf3-1a28f04b84ee admin admin] The token cache is being invalidate because project e120a75dce744ca78c4f36023901c711 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:16:20.891901 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-35f2a648-0a5c-4e5f-8cf3-1a28f04b84ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:20.892604 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-35f2a648-0a5c-4e5f-8cf3-1a28f04b84ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:20.894191 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1343/2698] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:16:20 2022] DELETE /identity/v3/projects/e120a75dce744ca78c4f36023901c711 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:16:20.898666 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6b34c-abd4-4cb4-b9c9-3afdbc496d57 service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:20.911381 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6b34c-abd4-4cb4-b9c9-3afdbc496d57 service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:20.985087 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22f456-9ad7-4555-8366-eaa138907734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:20.985754 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22f456-9ad7-4555-8366-eaa138907734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:20.986032 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22f456-9ad7-4555-8366-eaa138907734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:20.986307 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22f456-9ad7-4555-8366-eaa138907734 admin admin] PATH_INFO: `/v3/users/98a988be948e4d53abbbd673f9c971ae` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:20.987479 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab22f456-9ad7-4555-8366-eaa138907734 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98a988be948e4d53abbbd673f9c971ae)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:20.989402 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab22f456-9ad7-4555-8366-eaa138907734 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:21.006028 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:21.044528 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ab22f456-9ad7-4555-8366-eaa138907734 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98a988be948e4d53abbbd673f9c971ae'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:21.054874 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ab22f456-9ad7-4555-8366-eaa138907734 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98a988be948e4d53abbbd673f9c971ae'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:21.064740 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ab22f456-9ad7-4555-8366-eaa138907734 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98a988be948e4d53abbbd673f9c971ae'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:21.078506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ab22f456-9ad7-4555-8366-eaa138907734 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98a988be948e4d53abbbd673f9c971ae'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:21.079384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-ab22f456-9ad7-4555-8366-eaa138907734 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98a988be948e4d53abbbd673f9c971ae'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:21.085642 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1356/2699] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:16:20 2022] DELETE /identity/v3/users/98a988be948e4d53abbbd673f9c971ae => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:16:21.435387 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2a648-0a5c-4e5f-8cf3-1a28f04b84ee admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:21.437787 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-35f2a648-0a5c-4e5f-8cf3-1a28f04b84ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:21.454704 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2a648-0a5c-4e5f-8cf3-1a28f04b84ee admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:21.464344 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:21.561099 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762db544-45a6-4b13-93df-bcf738309e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:21.561769 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762db544-45a6-4b13-93df-bcf738309e71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:21.562355 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762db544-45a6-4b13-93df-bcf738309e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:21.562355 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762db544-45a6-4b13-93df-bcf738309e71 admin admin] PATH_INFO: `/v3/projects/30cdbf7283374804a84b35faae8089da` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:21.564869 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762db544-45a6-4b13-93df-bcf738309e71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30cdbf7283374804a84b35faae8089da)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:21.565420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762db544-45a6-4b13-93df-bcf738309e71 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:21.628553 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-762db544-45a6-4b13-93df-bcf738309e71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30cdbf7283374804a84b35faae8089da', 'request_id': 'req-762db544-45a6-4b13-93df-bcf738309e71'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:21.637788 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-762db544-45a6-4b13-93df-bcf738309e71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30cdbf7283374804a84b35faae8089da', 'request_id': 'req-762db544-45a6-4b13-93df-bcf738309e71'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:21.645204 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-762db544-45a6-4b13-93df-bcf738309e71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30cdbf7283374804a84b35faae8089da', 'request_id': 'req-762db544-45a6-4b13-93df-bcf738309e71'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:21.658613 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-762db544-45a6-4b13-93df-bcf738309e71 admin admin] The token cache is being invalidate because project 30cdbf7283374804a84b35faae8089da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:16:21.658862 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-762db544-45a6-4b13-93df-bcf738309e71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:21.659569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-762db544-45a6-4b13-93df-bcf738309e71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:21.661153 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1344/2700] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:16:21 2022] DELETE /identity/v3/projects/30cdbf7283374804a84b35faae8089da => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:16:21.667676 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb970759-5aaa-4237-82a1-b5d20620cc46 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:21.668309 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb970759-5aaa-4237-82a1-b5d20620cc46 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:21.668309 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb970759-5aaa-4237-82a1-b5d20620cc46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:21.693496 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-eb970759-5aaa-4237-82a1-b5d20620cc46 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:21.695450 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-eb970759-5aaa-4237-82a1-b5d20620cc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:21.784299 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1357/2701] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:16:21.792712 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa6416-7b93-4a87-a967-301222f28e41 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:21.793510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa6416-7b93-4a87-a967-301222f28e41 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:21.793510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa6416-7b93-4a87-a967-301222f28e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:21.817794 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-8bfa6416-7b93-4a87-a967-301222f28e41 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:21.819009 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-8bfa6416-7b93-4a87-a967-301222f28e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:21.827894 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1345/2702] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:16:21.833004 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb970759-5aaa-4237-82a1-b5d20620cc46 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:21.844556 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb970759-5aaa-4237-82a1-b5d20620cc46 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:21.851812 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f361be-4044-43dd-8ebc-e7bc4db606e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:21.852832 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f361be-4044-43dd-8ebc-e7bc4db606e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:21.853274 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f361be-4044-43dd-8ebc-e7bc4db606e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:21.853702 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f361be-4044-43dd-8ebc-e7bc4db606e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:21.854685 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f361be-4044-43dd-8ebc-e7bc4db606e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:16:21.855140 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f361be-4044-43dd-8ebc-e7bc4db606e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:21.857354 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f361be-4044-43dd-8ebc-e7bc4db606e1 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:21.873628 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1358/2703] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:16:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 20:16:21.879443 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa6416-7b93-4a87-a967-301222f28e41 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:21.890032 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa6416-7b93-4a87-a967-301222f28e41 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:21.898075 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db45de-f255-4d8f-9693-1de4e452f4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:21.898702 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db45de-f255-4d8f-9693-1de4e452f4ad admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:21.898981 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db45de-f255-4d8f-9693-1de4e452f4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:21.899255 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db45de-f255-4d8f-9693-1de4e452f4ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:21.899942 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db45de-f255-4d8f-9693-1de4e452f4ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:21.901862 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db45de-f255-4d8f-9693-1de4e452f4ad admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:21.939443 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1346/2704] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:16:21 2022] POST /identity/v3/projects => generated 379 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:16:21.945025 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f361be-4044-43dd-8ebc-e7bc4db606e1 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:21.959063 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f361be-4044-43dd-8ebc-e7bc4db606e1 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:22.027744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926cece-d898-4df6-a0c4-d2003e109ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:22.028363 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926cece-d898-4df6-a0c4-d2003e109ad8 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:22.028678 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926cece-d898-4df6-a0c4-d2003e109ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:22.028955 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926cece-d898-4df6-a0c4-d2003e109ad8 admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:22.029620 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926cece-d898-4df6-a0c4-d2003e109ad8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:22.032176 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926cece-d898-4df6-a0c4-d2003e109ad8 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:22.056441 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1359/2705] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:16:21 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:16:22.061110 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db45de-f255-4d8f-9693-1de4e452f4ad admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:22.072524 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db45de-f255-4d8f-9693-1de4e452f4ad admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:22.077435 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab7ca1-44a4-40d8-ba0c-20ad45084006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:22.078727 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab7ca1-44a4-40d8-ba0c-20ad45084006 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:22.079024 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab7ca1-44a4-40d8-ba0c-20ad45084006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:22.079314 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab7ca1-44a4-40d8-ba0c-20ad45084006 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:22.080025 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab7ca1-44a4-40d8-ba0c-20ad45084006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:22.081667 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab7ca1-44a4-40d8-ba0c-20ad45084006 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:22.092358 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1347/2706] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:16:22.097237 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926cece-d898-4df6-a0c4-d2003e109ad8 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:22.108918 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926cece-d898-4df6-a0c4-d2003e109ad8 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:22.116752 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71516ef4-7662-43f5-88fe-4e02b2adab2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:22.117512 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71516ef4-7662-43f5-88fe-4e02b2adab2f admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:22.117945 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71516ef4-7662-43f5-88fe-4e02b2adab2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:22.118310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71516ef4-7662-43f5-88fe-4e02b2adab2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:22.119225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71516ef4-7662-43f5-88fe-4e02b2adab2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:22.121505 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71516ef4-7662-43f5-88fe-4e02b2adab2f admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:22.130054 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:22.139482 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1360/2707] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:16:22.148915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab7ca1-44a4-40d8-ba0c-20ad45084006 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:22.160712 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab7ca1-44a4-40d8-ba0c-20ad45084006 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:22.165335 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554528a-f401-4ff2-af9d-d15661447c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:22.166002 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554528a-f401-4ff2-af9d-d15661447c81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:22.166280 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554528a-f401-4ff2-af9d-d15661447c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:22.166556 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554528a-f401-4ff2-af9d-d15661447c81 admin admin] PATH_INFO: `/v3/projects/6998c94952d3470bbc200e35630da791/users/68a9be714a944afda16205812d914257/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:22.182359 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554528a-f401-4ff2-af9d-d15661447c81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6998c94952d3470bbc200e35630da791, user_id=68a9be714a944afda16205812d914257, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:22.185035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554528a-f401-4ff2-af9d-d15661447c81 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:22.205374 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1348/2708] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:16:22 2022] PUT /identity/v3/projects/6998c94952d3470bbc200e35630da791/users/68a9be714a944afda16205812d914257/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:16:22.219048 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c529d-949a-4454-8705-7967b5de697d None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:22.219344 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c529d-949a-4454-8705-7967b5de697d None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:22.219652 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c529d-949a-4454-8705-7967b5de697d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:22.252123 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-e87c529d-949a-4454-8705-7967b5de697d None None] MFA Rules not processed for user `68a9be714a944afda16205812d914257`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:22.253376 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e87c529d-949a-4454-8705-7967b5de697d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:22.330748 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1361/2709] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:16:22.336369 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554528a-f401-4ff2-af9d-d15661447c81 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:22.384763 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:22.411104 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554528a-f401-4ff2-af9d-d15661447c81 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:22.416513 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a3512-05de-4a2c-b72d-67e61bb39cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:22.417154 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a3512-05de-4a2c-b72d-67e61bb39cd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:22.417422 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a3512-05de-4a2c-b72d-67e61bb39cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:22.417689 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a3512-05de-4a2c-b72d-67e61bb39cd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:22.418364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5a3512-05de-4a2c-b72d-67e61bb39cd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:22.420247 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5a3512-05de-4a2c-b72d-67e61bb39cd4 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:22.458309 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1349/2710] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:16:22 2022] POST /identity/v3/projects => generated 377 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:16:22.463733 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c529d-949a-4454-8705-7967b5de697d None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:22.479047 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c529d-949a-4454-8705-7967b5de697d None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:22.539115 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5daa6fe-4da9-49a2-a420-b938814b66fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:22.539729 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5daa6fe-4da9-49a2-a420-b938814b66fc admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:22.540002 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5daa6fe-4da9-49a2-a420-b938814b66fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:22.540272 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5daa6fe-4da9-49a2-a420-b938814b66fc admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:22.540967 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5daa6fe-4da9-49a2-a420-b938814b66fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:22.542873 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5daa6fe-4da9-49a2-a420-b938814b66fc admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:22.567018 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1362/2711] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:16:22 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:16:22.572468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a3512-05de-4a2c-b72d-67e61bb39cd4 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:22.583496 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a3512-05de-4a2c-b72d-67e61bb39cd4 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:22.590271 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c5e21-32e7-40d1-82f8-6f64ef5c9b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:22.590879 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c5e21-32e7-40d1-82f8-6f64ef5c9b8a admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:22.591159 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c5e21-32e7-40d1-82f8-6f64ef5c9b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:22.591433 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c5e21-32e7-40d1-82f8-6f64ef5c9b8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:22.592133 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667c5e21-32e7-40d1-82f8-6f64ef5c9b8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:22.594016 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667c5e21-32e7-40d1-82f8-6f64ef5c9b8a admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:22.605387 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1350/2712] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:16:22.610195 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5daa6fe-4da9-49a2-a420-b938814b66fc admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:22.631256 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5daa6fe-4da9-49a2-a420-b938814b66fc admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:22.636283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e72447f-b523-4377-8374-68e24a0072fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:22.636910 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72447f-b523-4377-8374-68e24a0072fa admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:22.637316 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72447f-b523-4377-8374-68e24a0072fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:22.637751 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72447f-b523-4377-8374-68e24a0072fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:22.638579 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e72447f-b523-4377-8374-68e24a0072fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:22.641018 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e72447f-b523-4377-8374-68e24a0072fa admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:22.654595 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1363/2713] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:16:22.660847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c5e21-32e7-40d1-82f8-6f64ef5c9b8a admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:22.671654 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c5e21-32e7-40d1-82f8-6f64ef5c9b8a admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:22.676938 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae5eaa-c39f-4484-8e07-94e0ea81f7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:22.677597 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae5eaa-c39f-4484-8e07-94e0ea81f7f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:22.677897 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae5eaa-c39f-4484-8e07-94e0ea81f7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:22.678284 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae5eaa-c39f-4484-8e07-94e0ea81f7f6 admin admin] PATH_INFO: `/v3/projects/34eb38d90571492ba87475f1fa89ff52/users/524b57611d7c43baa69889ddc7d33dee/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:22.694926 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae5eaa-c39f-4484-8e07-94e0ea81f7f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34eb38d90571492ba87475f1fa89ff52, user_id=524b57611d7c43baa69889ddc7d33dee, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:22.697604 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae5eaa-c39f-4484-8e07-94e0ea81f7f6 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:22.716791 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1351/2714] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:16:22 2022] PUT /identity/v3/projects/34eb38d90571492ba87475f1fa89ff52/users/524b57611d7c43baa69889ddc7d33dee/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:16:22.728680 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2baf6-4426-4d6a-b0de-c6927f32ed73 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:22.729268 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2baf6-4426-4d6a-b0de-c6927f32ed73 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:22.729660 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2baf6-4426-4d6a-b0de-c6927f32ed73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:22.760801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-9ac2baf6-4426-4d6a-b0de-c6927f32ed73 None None] MFA Rules not processed for user `524b57611d7c43baa69889ddc7d33dee`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:22.762958 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-9ac2baf6-4426-4d6a-b0de-c6927f32ed73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:22.846780 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:22.856150 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1364/2715] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:16:22.861698 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae5eaa-c39f-4484-8e07-94e0ea81f7f6 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:22.930476 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae5eaa-c39f-4484-8e07-94e0ea81f7f6 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:22.935214 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25283f40-3167-4b2a-9eaf-29217c86f86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:22.935830 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25283f40-3167-4b2a-9eaf-29217c86f86a admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:22.936119 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25283f40-3167-4b2a-9eaf-29217c86f86a admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:22.936393 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25283f40-3167-4b2a-9eaf-29217c86f86a admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:22.937134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25283f40-3167-4b2a-9eaf-29217c86f86a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:22.939803 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25283f40-3167-4b2a-9eaf-29217c86f86a admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:22.973761 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1352/2716] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:16:22 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:16:22.986384 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2baf6-4426-4d6a-b0de-c6927f32ed73 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:22.997688 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2baf6-4426-4d6a-b0de-c6927f32ed73 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:23.061091 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4683d4ae-2f25-4e1d-b5be-02bad308687e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:23.061727 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4683d4ae-2f25-4e1d-b5be-02bad308687e admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:23.062036 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4683d4ae-2f25-4e1d-b5be-02bad308687e admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:23.062948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4683d4ae-2f25-4e1d-b5be-02bad308687e admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:23.063723 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4683d4ae-2f25-4e1d-b5be-02bad308687e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:23.065796 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4683d4ae-2f25-4e1d-b5be-02bad308687e admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:23.089925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1365/2717] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:16:22 2022] POST /identity/v3/users => generated 295 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:16:23.094137 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25283f40-3167-4b2a-9eaf-29217c86f86a admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:23.105051 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25283f40-3167-4b2a-9eaf-29217c86f86a admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:23.110454 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589aee5d-46eb-4820-9cff-58924d9ec891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:23.111080 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589aee5d-46eb-4820-9cff-58924d9ec891 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:23.111357 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589aee5d-46eb-4820-9cff-58924d9ec891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:23.111629 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589aee5d-46eb-4820-9cff-58924d9ec891 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:23.112330 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589aee5d-46eb-4820-9cff-58924d9ec891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:23.114019 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589aee5d-46eb-4820-9cff-58924d9ec891 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:23.130794 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1353/2718] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:16:23.135822 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4683d4ae-2f25-4e1d-b5be-02bad308687e admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:23.147057 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4683d4ae-2f25-4e1d-b5be-02bad308687e admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:23.151953 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bd282-f5aa-4b77-bfde-3fe40b8abe3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:23.152693 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bd282-f5aa-4b77-bfde-3fe40b8abe3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:23.152928 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bd282-f5aa-4b77-bfde-3fe40b8abe3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:23.153201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bd282-f5aa-4b77-bfde-3fe40b8abe3a admin admin] PATH_INFO: `/v3/projects/3dca487e66c1470a98c876697f2a21fc/users/3f0af3ab09574ed2ba79a399a760a3cb/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:23.169053 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52bd282-f5aa-4b77-bfde-3fe40b8abe3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dca487e66c1470a98c876697f2a21fc, user_id=3f0af3ab09574ed2ba79a399a760a3cb, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:23.172535 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52bd282-f5aa-4b77-bfde-3fe40b8abe3a admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:23.189341 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1366/2719] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:16:23 2022] PUT /identity/v3/projects/3dca487e66c1470a98c876697f2a21fc/users/3f0af3ab09574ed2ba79a399a760a3cb/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:16:23.193515 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589aee5d-46eb-4820-9cff-58924d9ec891 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:23.207148 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589aee5d-46eb-4820-9cff-58924d9ec891 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:23.275610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d07f4-8751-4767-94f3-2c0ce39cd187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:23.276000 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d07f4-8751-4767-94f3-2c0ce39cd187 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:23.276284 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d07f4-8751-4767-94f3-2c0ce39cd187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:23.276568 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d07f4-8751-4767-94f3-2c0ce39cd187 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:23.277292 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6d07f4-8751-4767-94f3-2c0ce39cd187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:23.278898 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6d07f4-8751-4767-94f3-2c0ce39cd187 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:23.283129 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:23.298386 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1354/2720] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:16:23.303765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bd282-f5aa-4b77-bfde-3fe40b8abe3a admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:23.317561 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bd282-f5aa-4b77-bfde-3fe40b8abe3a admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:23.322842 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aee987-6965-4ec8-b054-a99d627635e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:23.323515 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aee987-6965-4ec8-b054-a99d627635e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:23.323797 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aee987-6965-4ec8-b054-a99d627635e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:23.324073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aee987-6965-4ec8-b054-a99d627635e2 admin admin] PATH_INFO: `/v3/projects/3dca487e66c1470a98c876697f2a21fc/users/3f0af3ab09574ed2ba79a399a760a3cb/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:23.326245 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aee987-6965-4ec8-b054-a99d627635e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dca487e66c1470a98c876697f2a21fc, user_id=3f0af3ab09574ed2ba79a399a760a3cb, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:23.330776 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aee987-6965-4ec8-b054-a99d627635e2 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:23.344196 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1367/2721] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:16:23 2022] PUT /identity/v3/projects/3dca487e66c1470a98c876697f2a21fc/users/3f0af3ab09574ed2ba79a399a760a3cb/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:16:23.364507 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67539bd2-7662-4580-93e7-13f1d3acfd16 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:23.364836 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67539bd2-7662-4580-93e7-13f1d3acfd16 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:23.365116 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67539bd2-7662-4580-93e7-13f1d3acfd16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:23.400103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-67539bd2-7662-4580-93e7-13f1d3acfd16 None None] MFA Rules not processed for user `3f0af3ab09574ed2ba79a399a760a3cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:23.401546 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-67539bd2-7662-4580-93e7-13f1d3acfd16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:23.509903 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1355/2722] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:23 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:16:23.528318 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aee987-6965-4ec8-b054-a99d627635e2 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:23.530074 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e8aee987-6965-4ec8-b054-a99d627635e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:23.623195 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aee987-6965-4ec8-b054-a99d627635e2 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:23.629629 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55811a7-9f28-4307-ac68-e12ae453f77a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:23.630525 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55811a7-9f28-4307-ac68-e12ae453f77a service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:23.630917 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55811a7-9f28-4307-ac68-e12ae453f77a service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:23.631329 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55811a7-9f28-4307-ac68-e12ae453f77a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:23.647493 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55811a7-9f28-4307-ac68-e12ae453f77a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:23.649223 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55811a7-9f28-4307-ac68-e12ae453f77a service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:23.659913 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1368/2723] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:16:23 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:16:23.706061 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67539bd2-7662-4580-93e7-13f1d3acfd16 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:23.707584 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-67539bd2-7662-4580-93e7-13f1d3acfd16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:23.726522 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67539bd2-7662-4580-93e7-13f1d3acfd16 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:23.735473 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d5ea76-e3af-4e5c-bb83-a751009832b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:23.736171 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5ea76-e3af-4e5c-bb83-a751009832b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:23.736451 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5ea76-e3af-4e5c-bb83-a751009832b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:23.736741 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5ea76-e3af-4e5c-bb83-a751009832b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:23.826483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d5ea76-e3af-4e5c-bb83-a751009832b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:23.829524 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d5ea76-e3af-4e5c-bb83-a751009832b9 service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:23.854822 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1356/2724] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:16:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:16:29.149235 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55811a7-9f28-4307-ac68-e12ae453f77a service cinder] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:29.218827 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:29.242494 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55811a7-9f28-4307-ac68-e12ae453f77a service cinder] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:29.250619 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765dd53f-9738-4de1-af39-055ad46e324b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:29.251707 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765dd53f-9738-4de1-af39-055ad46e324b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:29.252189 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765dd53f-9738-4de1-af39-055ad46e324b admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:29.252726 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765dd53f-9738-4de1-af39-055ad46e324b admin admin] PATH_INFO: `/v3/users/68a9be714a944afda16205812d914257` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:29.255305 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765dd53f-9738-4de1-af39-055ad46e324b admin admin] RBAC: Authorizing `identity:delete_user(user_id=68a9be714a944afda16205812d914257)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:29.259598 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765dd53f-9738-4de1-af39-055ad46e324b admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:29.322275 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-765dd53f-9738-4de1-af39-055ad46e324b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68a9be714a944afda16205812d914257'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:29.329569 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-765dd53f-9738-4de1-af39-055ad46e324b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68a9be714a944afda16205812d914257'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:29.337308 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-765dd53f-9738-4de1-af39-055ad46e324b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68a9be714a944afda16205812d914257'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:29.350533 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-765dd53f-9738-4de1-af39-055ad46e324b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68a9be714a944afda16205812d914257'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:29.351239 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-765dd53f-9738-4de1-af39-055ad46e324b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68a9be714a944afda16205812d914257'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:29.357182 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1369/2725] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:16:29 2022] DELETE /identity/v3/users/68a9be714a944afda16205812d914257 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:16:29.362453 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f33a1-7db7-453f-9b1a-a076a5a65651 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:29.363003 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f33a1-7db7-453f-9b1a-a076a5a65651 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:29.363003 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f33a1-7db7-453f-9b1a-a076a5a65651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:29.382029 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-559f33a1-7db7-453f-9b1a-a076a5a65651 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:29.383241 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-559f33a1-7db7-453f-9b1a-a076a5a65651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:29.445532 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1357/2726] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:16:29.456315 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765dd53f-9738-4de1-af39-055ad46e324b admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:29.457875 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-765dd53f-9738-4de1-af39-055ad46e324b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:29.522940 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765dd53f-9738-4de1-af39-055ad46e324b admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:29.527534 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec254b2e-1a54-447b-a578-25855402ebc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:29.528199 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec254b2e-1a54-447b-a578-25855402ebc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:29.528485 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec254b2e-1a54-447b-a578-25855402ebc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:29.528799 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec254b2e-1a54-447b-a578-25855402ebc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:29.539780 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec254b2e-1a54-447b-a578-25855402ebc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:29.541528 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec254b2e-1a54-447b-a578-25855402ebc7 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:29.553109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1370/2727] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:16:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:16:29.853188 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f33a1-7db7-453f-9b1a-a076a5a65651 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:29.854654 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-559f33a1-7db7-453f-9b1a-a076a5a65651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:29.868734 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f33a1-7db7-453f-9b1a-a076a5a65651 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:29.873362 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39476fc0-d27d-4947-9c94-ecbd224b42db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:29.873999 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39476fc0-d27d-4947-9c94-ecbd224b42db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:29.874277 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39476fc0-d27d-4947-9c94-ecbd224b42db admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:29.874648 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39476fc0-d27d-4947-9c94-ecbd224b42db admin admin] PATH_INFO: `/v3/projects/6998c94952d3470bbc200e35630da791` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:29.877367 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39476fc0-d27d-4947-9c94-ecbd224b42db admin admin] RBAC: Authorizing `identity:delete_project(project_id=6998c94952d3470bbc200e35630da791)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:29.879530 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39476fc0-d27d-4947-9c94-ecbd224b42db admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:29.898588 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:29.929847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-39476fc0-d27d-4947-9c94-ecbd224b42db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6998c94952d3470bbc200e35630da791', 'request_id': 'req-39476fc0-d27d-4947-9c94-ecbd224b42db'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:29.938046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-39476fc0-d27d-4947-9c94-ecbd224b42db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6998c94952d3470bbc200e35630da791', 'request_id': 'req-39476fc0-d27d-4947-9c94-ecbd224b42db'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:29.943738 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-39476fc0-d27d-4947-9c94-ecbd224b42db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6998c94952d3470bbc200e35630da791', 'request_id': 'req-39476fc0-d27d-4947-9c94-ecbd224b42db'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:29.955759 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-39476fc0-d27d-4947-9c94-ecbd224b42db admin admin] The token cache is being invalidate because project 6998c94952d3470bbc200e35630da791 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:16:29.956270 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-39476fc0-d27d-4947-9c94-ecbd224b42db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:29.956977 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-39476fc0-d27d-4947-9c94-ecbd224b42db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:29.958544 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1358/2728] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:16:29 2022] DELETE /identity/v3/projects/6998c94952d3470bbc200e35630da791 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:16:29.962629 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec254b2e-1a54-447b-a578-25855402ebc7 service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:29.973059 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec254b2e-1a54-447b-a578-25855402ebc7 service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:30.038027 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a4764-258d-4625-be34-6c41c69a88ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:30.038673 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a4764-258d-4625-be34-6c41c69a88ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:30.038948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a4764-258d-4625-be34-6c41c69a88ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:30.039221 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a4764-258d-4625-be34-6c41c69a88ab admin admin] PATH_INFO: `/v3/users/524b57611d7c43baa69889ddc7d33dee` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:30.040453 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a4764-258d-4625-be34-6c41c69a88ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=524b57611d7c43baa69889ddc7d33dee)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:30.042383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a4764-258d-4625-be34-6c41c69a88ab admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:30.058516 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:30.091123 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-bc9a4764-258d-4625-be34-6c41c69a88ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '524b57611d7c43baa69889ddc7d33dee'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:30.099813 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-bc9a4764-258d-4625-be34-6c41c69a88ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '524b57611d7c43baa69889ddc7d33dee'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:30.108057 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-bc9a4764-258d-4625-be34-6c41c69a88ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '524b57611d7c43baa69889ddc7d33dee'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:30.121317 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-bc9a4764-258d-4625-be34-6c41c69a88ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '524b57611d7c43baa69889ddc7d33dee'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:30.122045 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-bc9a4764-258d-4625-be34-6c41c69a88ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '524b57611d7c43baa69889ddc7d33dee'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:30.129610 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1371/2729] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:16:29 2022] DELETE /identity/v3/users/524b57611d7c43baa69889ddc7d33dee => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:16:30.312873 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39476fc0-d27d-4947-9c94-ecbd224b42db admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:30.314566 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-39476fc0-d27d-4947-9c94-ecbd224b42db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:30.329901 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39476fc0-d27d-4947-9c94-ecbd224b42db admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:30.400611 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775eb27-548e-4171-932a-ea35562209a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:30.401446 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775eb27-548e-4171-932a-ea35562209a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:30.401446 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775eb27-548e-4171-932a-ea35562209a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:30.401765 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775eb27-548e-4171-932a-ea35562209a4 admin admin] PATH_INFO: `/v3/users/a5d67da55a9d4541b3cc58ae8c80e6b3` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:30.402818 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775eb27-548e-4171-932a-ea35562209a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5d67da55a9d4541b3cc58ae8c80e6b3)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:30.404772 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775eb27-548e-4171-932a-ea35562209a4 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:30.448633 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a4764-258d-4625-be34-6c41c69a88ab admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:30.451300 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-bc9a4764-258d-4625-be34-6c41c69a88ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:30.462353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8775eb27-548e-4171-932a-ea35562209a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5d67da55a9d4541b3cc58ae8c80e6b3'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:30.474213 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8775eb27-548e-4171-932a-ea35562209a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5d67da55a9d4541b3cc58ae8c80e6b3'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:30.481050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a4764-258d-4625-be34-6c41c69a88ab admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:30.488982 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bfad8c-25be-46a4-93b3-f7e06dd1c783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:30.489670 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfad8c-25be-46a4-93b3-f7e06dd1c783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:30.489993 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfad8c-25be-46a4-93b3-f7e06dd1c783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:30.490239 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfad8c-25be-46a4-93b3-f7e06dd1c783 admin admin] PATH_INFO: `/v3/projects/34eb38d90571492ba87475f1fa89ff52` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:30.491189 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8775eb27-548e-4171-932a-ea35562209a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5d67da55a9d4541b3cc58ae8c80e6b3'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:30.491392 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bfad8c-25be-46a4-93b3-f7e06dd1c783 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34eb38d90571492ba87475f1fa89ff52)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:30.493103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8775eb27-548e-4171-932a-ea35562209a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5d67da55a9d4541b3cc58ae8c80e6b3'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:30.493407 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bfad8c-25be-46a4-93b3-f7e06dd1c783 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:30.503194 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8775eb27-548e-4171-932a-ea35562209a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5d67da55a9d4541b3cc58ae8c80e6b3'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:30.509290 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1359/2730] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:16:30 2022] DELETE /identity/v3/users/a5d67da55a9d4541b3cc58ae8c80e6b3 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:16:30.512447 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287178e6-5ccb-458e-8470-3a638af378ba None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:30.512840 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287178e6-5ccb-458e-8470-3a638af378ba None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:30.513122 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287178e6-5ccb-458e-8470-3a638af378ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:30.536118 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-287178e6-5ccb-458e-8470-3a638af378ba None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:30.537600 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-287178e6-5ccb-458e-8470-3a638af378ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:30.559224 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-d0bfad8c-25be-46a4-93b3-f7e06dd1c783 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34eb38d90571492ba87475f1fa89ff52', 'request_id': 'req-d0bfad8c-25be-46a4-93b3-f7e06dd1c783'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:30.560052 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:30.571238 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-d0bfad8c-25be-46a4-93b3-f7e06dd1c783 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34eb38d90571492ba87475f1fa89ff52', 'request_id': 'req-d0bfad8c-25be-46a4-93b3-f7e06dd1c783'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:30.582900 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-d0bfad8c-25be-46a4-93b3-f7e06dd1c783 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34eb38d90571492ba87475f1fa89ff52', 'request_id': 'req-d0bfad8c-25be-46a4-93b3-f7e06dd1c783'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:30.601172 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-d0bfad8c-25be-46a4-93b3-f7e06dd1c783 admin admin] The token cache is being invalidate because project 34eb38d90571492ba87475f1fa89ff52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:16:30.601472 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-d0bfad8c-25be-46a4-93b3-f7e06dd1c783 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:30.602127 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-d0bfad8c-25be-46a4-93b3-f7e06dd1c783 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:30.603639 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1372/2731] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:16:30 2022] DELETE /identity/v3/projects/34eb38d90571492ba87475f1fa89ff52 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:16:30.609810 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bfad8c-25be-46a4-93b3-f7e06dd1c783 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:30.611401 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-d0bfad8c-25be-46a4-93b3-f7e06dd1c783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:30.618735 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1360/2732] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:16:30.631003 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287178e6-5ccb-458e-8470-3a638af378ba None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:30.632825 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-287178e6-5ccb-458e-8470-3a638af378ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:30.683257 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bfad8c-25be-46a4-93b3-f7e06dd1c783 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:30.689061 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7bd772-2fac-47cf-b487-b87339289089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:30.689702 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7bd772-2fac-47cf-b487-b87339289089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:30.689980 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7bd772-2fac-47cf-b487-b87339289089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:30.690251 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7bd772-2fac-47cf-b487-b87339289089 admin admin] PATH_INFO: `/v3/users/3f0af3ab09574ed2ba79a399a760a3cb` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:30.691438 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7bd772-2fac-47cf-b487-b87339289089 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f0af3ab09574ed2ba79a399a760a3cb)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:30.693376 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7bd772-2fac-47cf-b487-b87339289089 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:30.705306 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287178e6-5ccb-458e-8470-3a638af378ba None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:30.716048 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:30.748193 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-7a7bd772-2fac-47cf-b487-b87339289089 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f0af3ab09574ed2ba79a399a760a3cb'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:30.759380 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-7a7bd772-2fac-47cf-b487-b87339289089 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f0af3ab09574ed2ba79a399a760a3cb'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:30.767721 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-7a7bd772-2fac-47cf-b487-b87339289089 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f0af3ab09574ed2ba79a399a760a3cb'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:30.768518 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c405140-92e3-4c3b-8f5d-9dc5039984b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:30.769498 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c405140-92e3-4c3b-8f5d-9dc5039984b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:30.770167 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c405140-92e3-4c3b-8f5d-9dc5039984b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:30.770696 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c405140-92e3-4c3b-8f5d-9dc5039984b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:30.780670 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c405140-92e3-4c3b-8f5d-9dc5039984b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:30.782737 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-7a7bd772-2fac-47cf-b487-b87339289089 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f0af3ab09574ed2ba79a399a760a3cb'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:30.783328 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c405140-92e3-4c3b-8f5d-9dc5039984b3 service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:30.783900 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-7a7bd772-2fac-47cf-b487-b87339289089 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f0af3ab09574ed2ba79a399a760a3cb'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:30.790298 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1373/2733] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:16:30 2022] DELETE /identity/v3/users/3f0af3ab09574ed2ba79a399a760a3cb => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:16:30.797846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1361/2734] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:16:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:16:31.170508 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7bd772-2fac-47cf-b487-b87339289089 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:31.173237 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-7a7bd772-2fac-47cf-b487-b87339289089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:31.210615 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c405140-92e3-4c3b-8f5d-9dc5039984b3 service neutron] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:31.212081 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-2c405140-92e3-4c3b-8f5d-9dc5039984b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:31.231163 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c405140-92e3-4c3b-8f5d-9dc5039984b3 service neutron] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:31.286793 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7bd772-2fac-47cf-b487-b87339289089 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:31.292251 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef833e-2c85-4ac4-b95e-303a79e0aa2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:31.293103 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef833e-2c85-4ac4-b95e-303a79e0aa2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:31.293470 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef833e-2c85-4ac4-b95e-303a79e0aa2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:31.293920 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef833e-2c85-4ac4-b95e-303a79e0aa2a admin admin] PATH_INFO: `/v3/projects/3dca487e66c1470a98c876697f2a21fc` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:31.295175 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ef833e-2c85-4ac4-b95e-303a79e0aa2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dca487e66c1470a98c876697f2a21fc)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:31.298066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ef833e-2c85-4ac4-b95e-303a79e0aa2a admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:31.307307 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b2c92-a29f-40b5-9312-d7f50fc4e7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:31.308047 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b2c92-a29f-40b5-9312-d7f50fc4e7e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:31.308343 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b2c92-a29f-40b5-9312-d7f50fc4e7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:31.308655 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b2c92-a29f-40b5-9312-d7f50fc4e7e6 admin admin] PATH_INFO: `/v3/projects/663b91426b32473ba74764447b10b7cd` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:31.310491 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b2c92-a29f-40b5-9312-d7f50fc4e7e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=663b91426b32473ba74764447b10b7cd)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:31.313153 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b2c92-a29f-40b5-9312-d7f50fc4e7e6 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:31.349582 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-11ef833e-2c85-4ac4-b95e-303a79e0aa2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dca487e66c1470a98c876697f2a21fc', 'request_id': 'req-11ef833e-2c85-4ac4-b95e-303a79e0aa2a'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:31.357337 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:31.358214 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-11ef833e-2c85-4ac4-b95e-303a79e0aa2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dca487e66c1470a98c876697f2a21fc', 'request_id': 'req-11ef833e-2c85-4ac4-b95e-303a79e0aa2a'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:31.369409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-11ef833e-2c85-4ac4-b95e-303a79e0aa2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dca487e66c1470a98c876697f2a21fc', 'request_id': 'req-11ef833e-2c85-4ac4-b95e-303a79e0aa2a'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:31.382564 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-027b2c92-a29f-40b5-9312-d7f50fc4e7e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '663b91426b32473ba74764447b10b7cd', 'request_id': 'req-027b2c92-a29f-40b5-9312-d7f50fc4e7e6'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:31.383533 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-11ef833e-2c85-4ac4-b95e-303a79e0aa2a admin admin] The token cache is being invalidate because project 3dca487e66c1470a98c876697f2a21fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:16:31.384057 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-11ef833e-2c85-4ac4-b95e-303a79e0aa2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:31.384901 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-11ef833e-2c85-4ac4-b95e-303a79e0aa2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:31.386961 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1374/2735] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:16:31 2022] DELETE /identity/v3/projects/3dca487e66c1470a98c876697f2a21fc => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:16:31.392494 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-027b2c92-a29f-40b5-9312-d7f50fc4e7e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '663b91426b32473ba74764447b10b7cd', 'request_id': 'req-027b2c92-a29f-40b5-9312-d7f50fc4e7e6'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:31.394295 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a29a48-aff9-473e-82d2-e0bd0c0b06f7 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:31.395043 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a29a48-aff9-473e-82d2-e0bd0c0b06f7 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:31.395687 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a29a48-aff9-473e-82d2-e0bd0c0b06f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:31.400614 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-027b2c92-a29f-40b5-9312-d7f50fc4e7e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '663b91426b32473ba74764447b10b7cd', 'request_id': 'req-027b2c92-a29f-40b5-9312-d7f50fc4e7e6'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:31.414423 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-027b2c92-a29f-40b5-9312-d7f50fc4e7e6 admin admin] The token cache is being invalidate because project 663b91426b32473ba74764447b10b7cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:16:31.415052 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-027b2c92-a29f-40b5-9312-d7f50fc4e7e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:31.415564 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-027b2c92-a29f-40b5-9312-d7f50fc4e7e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:31.416985 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-d4a29a48-aff9-473e-82d2-e0bd0c0b06f7 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:31.417366 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1362/2736] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:16:31 2022] DELETE /identity/v3/projects/663b91426b32473ba74764447b10b7cd => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:16:31.425357 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-d4a29a48-aff9-473e-82d2-e0bd0c0b06f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:31.446961 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:31.518080 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1375/2737] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:16:31.533530 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed25ecc-64ef-42b6-852f-4db68227cda0 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:31.533796 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed25ecc-64ef-42b6-852f-4db68227cda0 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:31.534074 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed25ecc-64ef-42b6-852f-4db68227cda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:31.556469 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-0ed25ecc-64ef-42b6-852f-4db68227cda0 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:31.557879 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-0ed25ecc-64ef-42b6-852f-4db68227cda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:31.566884 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1363/2738] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:16:31.571383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a29a48-aff9-473e-82d2-e0bd0c0b06f7 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:31.582163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a29a48-aff9-473e-82d2-e0bd0c0b06f7 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:31.587703 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f95d7-e843-41c2-8d5e-79687f3fb56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:31.588350 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f95d7-e843-41c2-8d5e-79687f3fb56b admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:31.588671 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f95d7-e843-41c2-8d5e-79687f3fb56b admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:31.588954 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f95d7-e843-41c2-8d5e-79687f3fb56b admin admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:31.589676 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f95d7-e843-41c2-8d5e-79687f3fb56b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:16:31.589997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f95d7-e843-41c2-8d5e-79687f3fb56b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:31.591662 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f95d7-e843-41c2-8d5e-79687f3fb56b admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:31.611530 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1376/2739] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:16:31.612952 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed25ecc-64ef-42b6-852f-4db68227cda0 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:31.622668 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed25ecc-64ef-42b6-852f-4db68227cda0 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:31.628326 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5c42e-ab01-4467-ae93-b3c756db861a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:31.629012 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5c42e-ab01-4467-ae93-b3c756db861a admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:31.629294 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5c42e-ab01-4467-ae93-b3c756db861a admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:31.629568 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5c42e-ab01-4467-ae93-b3c756db861a admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:31.630256 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b5c42e-ab01-4467-ae93-b3c756db861a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:31.632208 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b5c42e-ab01-4467-ae93-b3c756db861a admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:31.665515 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1364/2740] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:16:31 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:16:31.669994 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f95d7-e843-41c2-8d5e-79687f3fb56b admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:31.681709 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f95d7-e843-41c2-8d5e-79687f3fb56b admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:31.754618 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f953c-8bc3-4d1a-859e-f365cf14ca1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:31.755413 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f953c-8bc3-4d1a-859e-f365cf14ca1e admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:31.755724 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f953c-8bc3-4d1a-859e-f365cf14ca1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:31.755724 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f953c-8bc3-4d1a-859e-f365cf14ca1e admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:31.756409 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f953c-8bc3-4d1a-859e-f365cf14ca1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:31.758332 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f953c-8bc3-4d1a-859e-f365cf14ca1e admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:31.781334 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1377/2741] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:16:31 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:16:31.790884 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5c42e-ab01-4467-ae93-b3c756db861a admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:31.801556 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5c42e-ab01-4467-ae93-b3c756db861a admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:31.806419 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910dc39-1d7e-4da4-abd2-4db3fdca0a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:31.807038 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910dc39-1d7e-4da4-abd2-4db3fdca0a44 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:31.807319 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910dc39-1d7e-4da4-abd2-4db3fdca0a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:31.807610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910dc39-1d7e-4da4-abd2-4db3fdca0a44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:31.808306 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a910dc39-1d7e-4da4-abd2-4db3fdca0a44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:31.809949 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a910dc39-1d7e-4da4-abd2-4db3fdca0a44 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:31.820735 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1365/2742] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:16:31.824947 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f953c-8bc3-4d1a-859e-f365cf14ca1e admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:31.837416 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f953c-8bc3-4d1a-859e-f365cf14ca1e admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:31.842754 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783adcc2-80fc-4452-b54f-cb186123300d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:31.843823 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783adcc2-80fc-4452-b54f-cb186123300d admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:31.844110 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783adcc2-80fc-4452-b54f-cb186123300d admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:31.844476 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783adcc2-80fc-4452-b54f-cb186123300d admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:31.845350 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783adcc2-80fc-4452-b54f-cb186123300d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:31.846961 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783adcc2-80fc-4452-b54f-cb186123300d admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:31.859872 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1378/2743] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:16:31.864973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910dc39-1d7e-4da4-abd2-4db3fdca0a44 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:31.878236 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910dc39-1d7e-4da4-abd2-4db3fdca0a44 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:31.883353 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec48e51-e163-4a72-8270-e0744b0ee4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:31.884061 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec48e51-e163-4a72-8270-e0744b0ee4ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:31.884338 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec48e51-e163-4a72-8270-e0744b0ee4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:31.884640 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec48e51-e163-4a72-8270-e0744b0ee4ea admin admin] PATH_INFO: `/v3/projects/13b1a095aff0459fa0f7b1f9b2449798/users/f84e8505f1ee470887346ef7442c9a72/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:31.902361 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec48e51-e163-4a72-8270-e0744b0ee4ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=13b1a095aff0459fa0f7b1f9b2449798, user_id=f84e8505f1ee470887346ef7442c9a72, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:31.905103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec48e51-e163-4a72-8270-e0744b0ee4ea admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:31.924698 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1366/2744] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:16:31 2022] PUT /identity/v3/projects/13b1a095aff0459fa0f7b1f9b2449798/users/f84e8505f1ee470887346ef7442c9a72/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:16:31.991413 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e173bb5a-a7b1-448b-bf12-e8be6c25ee00 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:31.991735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e173bb5a-a7b1-448b-bf12-e8be6c25ee00 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:31.992129 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e173bb5a-a7b1-448b-bf12-e8be6c25ee00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:32.022749 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-e173bb5a-a7b1-448b-bf12-e8be6c25ee00 None None] MFA Rules not processed for user `f84e8505f1ee470887346ef7442c9a72`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:32.023970 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e173bb5a-a7b1-448b-bf12-e8be6c25ee00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:32.100770 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1379/2745] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:16:32.113784 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec48e51-e163-4a72-8270-e0744b0ee4ea admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:32.115589 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-5ec48e51-e163-4a72-8270-e0744b0ee4ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:32.182122 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec48e51-e163-4a72-8270-e0744b0ee4ea admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:32.185654 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556dc3d3-d202-4792-9aef-172ac3065e4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:32.186265 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dc3d3-d202-4792-9aef-172ac3065e4a service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:32.186545 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dc3d3-d202-4792-9aef-172ac3065e4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:32.186820 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dc3d3-d202-4792-9aef-172ac3065e4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:32.201890 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556dc3d3-d202-4792-9aef-172ac3065e4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:32.204090 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556dc3d3-d202-4792-9aef-172ac3065e4a service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:32.220782 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1367/2746] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:16:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:16:34.806813 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e173bb5a-a7b1-448b-bf12-e8be6c25ee00 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:34.835533 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:34.879170 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e173bb5a-a7b1-448b-bf12-e8be6c25ee00 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:34.883965 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc509c72-8cff-4f71-ae23-a50e7c505b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:34.884624 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc509c72-8cff-4f71-ae23-a50e7c505b20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:34.885058 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc509c72-8cff-4f71-ae23-a50e7c505b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:34.885331 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc509c72-8cff-4f71-ae23-a50e7c505b20 admin admin] PATH_INFO: `/v3/users/f84e8505f1ee470887346ef7442c9a72` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:34.886487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc509c72-8cff-4f71-ae23-a50e7c505b20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f84e8505f1ee470887346ef7442c9a72)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:34.888387 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc509c72-8cff-4f71-ae23-a50e7c505b20 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:34.937718 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-bc509c72-8cff-4f71-ae23-a50e7c505b20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f84e8505f1ee470887346ef7442c9a72'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:34.946332 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-bc509c72-8cff-4f71-ae23-a50e7c505b20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f84e8505f1ee470887346ef7442c9a72'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:34.953458 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-bc509c72-8cff-4f71-ae23-a50e7c505b20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f84e8505f1ee470887346ef7442c9a72'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:34.965269 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-bc509c72-8cff-4f71-ae23-a50e7c505b20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f84e8505f1ee470887346ef7442c9a72'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:34.965957 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-bc509c72-8cff-4f71-ae23-a50e7c505b20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f84e8505f1ee470887346ef7442c9a72'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:34.971791 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1380/2747] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:16:34 2022] DELETE /identity/v3/users/f84e8505f1ee470887346ef7442c9a72 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:16:34.976299 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006cb61-2174-4cf4-892b-94152cd5ef9d None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:34.977009 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006cb61-2174-4cf4-892b-94152cd5ef9d None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:34.977009 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006cb61-2174-4cf4-892b-94152cd5ef9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:34.998687 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-4006cb61-2174-4cf4-892b-94152cd5ef9d None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:35.002374 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-4006cb61-2174-4cf4-892b-94152cd5ef9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:35.050405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:35.066478 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1368/2748] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:16:35.077016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc509c72-8cff-4f71-ae23-a50e7c505b20 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:35.078977 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-bc509c72-8cff-4f71-ae23-a50e7c505b20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:35.136520 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc509c72-8cff-4f71-ae23-a50e7c505b20 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:35.140767 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93d309c-7f6b-4291-a315-cc935ecd7c37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:35.141340 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d309c-7f6b-4291-a315-cc935ecd7c37 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:35.141605 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d309c-7f6b-4291-a315-cc935ecd7c37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:35.141874 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d309c-7f6b-4291-a315-cc935ecd7c37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:35.152141 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93d309c-7f6b-4291-a315-cc935ecd7c37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:35.153598 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93d309c-7f6b-4291-a315-cc935ecd7c37 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:35.163580 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1381/2749] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:16:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:16:35.449712 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006cb61-2174-4cf4-892b-94152cd5ef9d None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:35.452358 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-4006cb61-2174-4cf4-892b-94152cd5ef9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:35.472462 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006cb61-2174-4cf4-892b-94152cd5ef9d None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:35.477418 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb2288-363f-4145-b2c8-eb12190ffbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:35.478069 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb2288-363f-4145-b2c8-eb12190ffbc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:35.478351 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb2288-363f-4145-b2c8-eb12190ffbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:35.478718 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb2288-363f-4145-b2c8-eb12190ffbc6 admin admin] PATH_INFO: `/v3/projects/13b1a095aff0459fa0f7b1f9b2449798` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:35.479816 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb2288-363f-4145-b2c8-eb12190ffbc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13b1a095aff0459fa0f7b1f9b2449798)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:35.481759 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb2288-363f-4145-b2c8-eb12190ffbc6 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:35.543834 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-97cb2288-363f-4145-b2c8-eb12190ffbc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13b1a095aff0459fa0f7b1f9b2449798', 'request_id': 'req-97cb2288-363f-4145-b2c8-eb12190ffbc6'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:35.555560 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-97cb2288-363f-4145-b2c8-eb12190ffbc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13b1a095aff0459fa0f7b1f9b2449798', 'request_id': 'req-97cb2288-363f-4145-b2c8-eb12190ffbc6'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:35.564912 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-97cb2288-363f-4145-b2c8-eb12190ffbc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13b1a095aff0459fa0f7b1f9b2449798', 'request_id': 'req-97cb2288-363f-4145-b2c8-eb12190ffbc6'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:35.583757 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-97cb2288-363f-4145-b2c8-eb12190ffbc6 admin admin] The token cache is being invalidate because project 13b1a095aff0459fa0f7b1f9b2449798 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:16:35.584102 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-97cb2288-363f-4145-b2c8-eb12190ffbc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:35.584755 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-97cb2288-363f-4145-b2c8-eb12190ffbc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:35.586324 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1369/2750] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:16:35 2022] DELETE /identity/v3/projects/13b1a095aff0459fa0f7b1f9b2449798 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:16:35.592729 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effadd6e-db90-4ecc-a3c0-bdd07c065e82 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:35.593132 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effadd6e-db90-4ecc-a3c0-bdd07c065e82 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:35.593537 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effadd6e-db90-4ecc-a3c0-bdd07c065e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:35.617673 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-effadd6e-db90-4ecc-a3c0-bdd07c065e82 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:35.618896 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-effadd6e-db90-4ecc-a3c0-bdd07c065e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:35.662945 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:35.684762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1382/2751] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:16:35.693662 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481cc794-f895-489f-9ff4-e050f72a4f33 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:35.693955 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481cc794-f895-489f-9ff4-e050f72a4f33 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:35.694235 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481cc794-f895-489f-9ff4-e050f72a4f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:35.713918 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-481cc794-f895-489f-9ff4-e050f72a4f33 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:35.715130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-481cc794-f895-489f-9ff4-e050f72a4f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:35.723385 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1370/2752] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:16:35.727736 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effadd6e-db90-4ecc-a3c0-bdd07c065e82 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:35.738069 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effadd6e-db90-4ecc-a3c0-bdd07c065e82 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:35.743735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe7198-5300-49fd-bd76-e02f4f9ab863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:35.744336 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe7198-5300-49fd-bd76-e02f4f9ab863 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:35.744713 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe7198-5300-49fd-bd76-e02f4f9ab863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:35.744991 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe7198-5300-49fd-bd76-e02f4f9ab863 admin admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:35.745722 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe7198-5300-49fd-bd76-e02f4f9ab863 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:16:35.745989 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe7198-5300-49fd-bd76-e02f4f9ab863 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:35.747642 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe7198-5300-49fd-bd76-e02f4f9ab863 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:35.762550 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1383/2753] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:16:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:16:35.767117 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481cc794-f895-489f-9ff4-e050f72a4f33 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:35.777720 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481cc794-f895-489f-9ff4-e050f72a4f33 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:35.782846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0a75a-b2fb-4b4c-ac0c-af26afaaa820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:35.783458 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0a75a-b2fb-4b4c-ac0c-af26afaaa820 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:35.783733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0a75a-b2fb-4b4c-ac0c-af26afaaa820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:35.784010 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0a75a-b2fb-4b4c-ac0c-af26afaaa820 admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:35.784713 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0a75a-b2fb-4b4c-ac0c-af26afaaa820 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:35.786602 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0a75a-b2fb-4b4c-ac0c-af26afaaa820 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:35.817190 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1371/2754] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:16:35 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:16:35.823546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe7198-5300-49fd-bd76-e02f4f9ab863 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:35.837080 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe7198-5300-49fd-bd76-e02f4f9ab863 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:35.902784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dccefc-83a0-4d0d-88ae-4b8611bc139f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:35.903408 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dccefc-83a0-4d0d-88ae-4b8611bc139f admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:35.903732 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dccefc-83a0-4d0d-88ae-4b8611bc139f admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:35.903993 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dccefc-83a0-4d0d-88ae-4b8611bc139f admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:35.904726 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dccefc-83a0-4d0d-88ae-4b8611bc139f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:35.906864 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dccefc-83a0-4d0d-88ae-4b8611bc139f admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:35.928462 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1384/2755] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:16:35 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:16:35.932729 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0a75a-b2fb-4b4c-ac0c-af26afaaa820 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:35.943942 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0a75a-b2fb-4b4c-ac0c-af26afaaa820 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:35.948714 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed983314-4cd9-44d1-ba62-60f9d9a913fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:35.949320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed983314-4cd9-44d1-ba62-60f9d9a913fd admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:35.949602 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed983314-4cd9-44d1-ba62-60f9d9a913fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:35.949875 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed983314-4cd9-44d1-ba62-60f9d9a913fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:35.950564 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed983314-4cd9-44d1-ba62-60f9d9a913fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:35.952151 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed983314-4cd9-44d1-ba62-60f9d9a913fd admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:35.962875 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1372/2756] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:16:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:16:35.967005 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dccefc-83a0-4d0d-88ae-4b8611bc139f admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:35.981536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dccefc-83a0-4d0d-88ae-4b8611bc139f admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:35.988012 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53a766-36a2-4c7b-92e3-c41dd6580745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:35.988012 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53a766-36a2-4c7b-92e3-c41dd6580745 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:35.988545 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53a766-36a2-4c7b-92e3-c41dd6580745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:35.988545 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53a766-36a2-4c7b-92e3-c41dd6580745 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:35.990781 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53a766-36a2-4c7b-92e3-c41dd6580745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:35.990781 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53a766-36a2-4c7b-92e3-c41dd6580745 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:36.002212 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1385/2757] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:16:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:16:36.007084 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed983314-4cd9-44d1-ba62-60f9d9a913fd admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:36.017805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed983314-4cd9-44d1-ba62-60f9d9a913fd admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:36.023112 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f22cc-8c0a-4b4f-9855-d61c812db27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:36.023774 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f22cc-8c0a-4b4f-9855-d61c812db27f admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:36.024051 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f22cc-8c0a-4b4f-9855-d61c812db27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:36.024326 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f22cc-8c0a-4b4f-9855-d61c812db27f admin admin] PATH_INFO: `/v3/projects/a07079d8ffff45ae90f57c90aec00915/users/2efa43a179b349d2b56eb9bca989185a/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:36.041371 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75f22cc-8c0a-4b4f-9855-d61c812db27f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a07079d8ffff45ae90f57c90aec00915, user_id=2efa43a179b349d2b56eb9bca989185a, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:36.044246 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75f22cc-8c0a-4b4f-9855-d61c812db27f admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:36.060967 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1373/2758] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:16:36 2022] PUT /identity/v3/projects/a07079d8ffff45ae90f57c90aec00915/users/2efa43a179b349d2b56eb9bca989185a/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:16:36.075451 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd89daf-1b68-4684-842d-46e9d0b34694 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:36.075740 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd89daf-1b68-4684-842d-46e9d0b34694 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:36.076017 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd89daf-1b68-4684-842d-46e9d0b34694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:36.110153 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-1bd89daf-1b68-4684-842d-46e9d0b34694 None None] MFA Rules not processed for user `2efa43a179b349d2b56eb9bca989185a`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:36.111368 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-1bd89daf-1b68-4684-842d-46e9d0b34694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:36.206402 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1386/2759] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:16:36.217433 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f22cc-8c0a-4b4f-9855-d61c812db27f admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:36.222663 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-c75f22cc-8c0a-4b4f-9855-d61c812db27f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:36.294659 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f22cc-8c0a-4b4f-9855-d61c812db27f admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:36.299135 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc2d3b-8897-4391-a34e-71cc4c512285 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:36.299760 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc2d3b-8897-4391-a34e-71cc4c512285 service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:36.300042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc2d3b-8897-4391-a34e-71cc4c512285 service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:36.300313 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc2d3b-8897-4391-a34e-71cc4c512285 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:36.309349 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bc2d3b-8897-4391-a34e-71cc4c512285 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:36.310776 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bc2d3b-8897-4391-a34e-71cc4c512285 service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:36.319960 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1374/2760] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:16:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:16:46.206506 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=79,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:46.315963 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:47.166377 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd89daf-1b68-4684-842d-46e9d0b34694 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:47.228862 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd89daf-1b68-4684-842d-46e9d0b34694 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:47.233740 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645d34e-9635-46e9-bb80-c0b14c399608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:47.234472 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645d34e-9635-46e9-bb80-c0b14c399608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:47.234751 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645d34e-9635-46e9-bb80-c0b14c399608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:47.235026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645d34e-9635-46e9-bb80-c0b14c399608 admin admin] PATH_INFO: `/v3/users/2efa43a179b349d2b56eb9bca989185a` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:47.236222 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2645d34e-9635-46e9-bb80-c0b14c399608 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2efa43a179b349d2b56eb9bca989185a)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:47.238203 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2645d34e-9635-46e9-bb80-c0b14c399608 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:47.289526 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-2645d34e-9635-46e9-bb80-c0b14c399608 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2efa43a179b349d2b56eb9bca989185a'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:47.297633 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-2645d34e-9635-46e9-bb80-c0b14c399608 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2efa43a179b349d2b56eb9bca989185a'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:47.304750 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-2645d34e-9635-46e9-bb80-c0b14c399608 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2efa43a179b349d2b56eb9bca989185a'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:47.317733 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-2645d34e-9635-46e9-bb80-c0b14c399608 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2efa43a179b349d2b56eb9bca989185a'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:47.318574 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-2645d34e-9635-46e9-bb80-c0b14c399608 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2efa43a179b349d2b56eb9bca989185a'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:47.324502 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1387/2761] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:16:47 2022] DELETE /identity/v3/users/2efa43a179b349d2b56eb9bca989185a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:16:47.330330 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3301937-fc74-4e34-b9e0-9728cd296e4f None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:47.330844 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3301937-fc74-4e34-b9e0-9728cd296e4f None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:47.330950 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3301937-fc74-4e34-b9e0-9728cd296e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:47.351296 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-f3301937-fc74-4e34-b9e0-9728cd296e4f None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:16:47.352641 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-f3301937-fc74-4e34-b9e0-9728cd296e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:47.413865 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1375/2762] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:16:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:16:47.424267 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645d34e-9635-46e9-bb80-c0b14c399608 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:47.425733 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-2645d34e-9635-46e9-bb80-c0b14c399608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:47.485421 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645d34e-9635-46e9-bb80-c0b14c399608 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:47.489085 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d88d88-7505-44fe-a02c-3ee1bb01d9b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:47.489686 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d88d88-7505-44fe-a02c-3ee1bb01d9b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:47.489960 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d88d88-7505-44fe-a02c-3ee1bb01d9b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:47.490230 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d88d88-7505-44fe-a02c-3ee1bb01d9b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:47.500285 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d88d88-7505-44fe-a02c-3ee1bb01d9b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:47.501865 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d88d88-7505-44fe-a02c-3ee1bb01d9b9 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:47.516517 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1388/2763] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:16:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:16:47.814010 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3301937-fc74-4e34-b9e0-9728cd296e4f None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:16:47.815508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-f3301937-fc74-4e34-b9e0-9728cd296e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:16:47.827979 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3301937-fc74-4e34-b9e0-9728cd296e4f None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:16:47.833607 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a691ddd-a803-4c90-b4a9-da7a4fca007c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:16:47.834333 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a691ddd-a803-4c90-b4a9-da7a4fca007c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:16:47.834701 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a691ddd-a803-4c90-b4a9-da7a4fca007c admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:16:47.835046 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a691ddd-a803-4c90-b4a9-da7a4fca007c admin admin] PATH_INFO: `/v3/projects/a07079d8ffff45ae90f57c90aec00915` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:16:47.836525 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a691ddd-a803-4c90-b4a9-da7a4fca007c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a07079d8ffff45ae90f57c90aec00915)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:16:47.838561 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a691ddd-a803-4c90-b4a9-da7a4fca007c admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:16:47.888113 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8a691ddd-a803-4c90-b4a9-da7a4fca007c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a07079d8ffff45ae90f57c90aec00915', 'request_id': 'req-8a691ddd-a803-4c90-b4a9-da7a4fca007c'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:47.896161 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8a691ddd-a803-4c90-b4a9-da7a4fca007c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a07079d8ffff45ae90f57c90aec00915', 'request_id': 'req-8a691ddd-a803-4c90-b4a9-da7a4fca007c'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:47.901875 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8a691ddd-a803-4c90-b4a9-da7a4fca007c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a07079d8ffff45ae90f57c90aec00915', 'request_id': 'req-8a691ddd-a803-4c90-b4a9-da7a4fca007c'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:47.914162 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8a691ddd-a803-4c90-b4a9-da7a4fca007c admin admin] The token cache is being invalidate because project a07079d8ffff45ae90f57c90aec00915 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:16:47.914444 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8a691ddd-a803-4c90-b4a9-da7a4fca007c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:47.915104 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8a691ddd-a803-4c90-b4a9-da7a4fca007c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:16:47.916631 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1376/2764] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:16:47 2022] DELETE /identity/v3/projects/a07079d8ffff45ae90f57c90aec00915 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:16:57.511214 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:16:57.910982 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:17:11.415669 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d88d88-7505-44fe-a02c-3ee1bb01d9b9 service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:17:11.417087 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-82d88d88-7505-44fe-a02c-3ee1bb01d9b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:17:11.477262 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d88d88-7505-44fe-a02c-3ee1bb01d9b9 service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:17:11.527592 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c3d28-f146-432d-bc16-a03200e7ce6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:17:11.528238 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c3d28-f146-432d-bc16-a03200e7ce6a service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:17:11.528516 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c3d28-f146-432d-bc16-a03200e7ce6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:17:11.528824 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c3d28-f146-432d-bc16-a03200e7ce6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:17:11.530631 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-b77c3d28-f146-432d-bc16-a03200e7ce6a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:17:11.589040 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77c3d28-f146-432d-bc16-a03200e7ce6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:17:11.590494 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77c3d28-f146-432d-bc16-a03200e7ce6a service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:17:11.600093 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1389/2765] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:17:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:17:18.822014 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2981c09-4571-4f9b-9b9c-c7900fbc6ffc None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:17:18.822616 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2981c09-4571-4f9b-9b9c-c7900fbc6ffc None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:17:18.822616 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2981c09-4571-4f9b-9b9c-c7900fbc6ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:17:18.841756 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-d2981c09-4571-4f9b-9b9c-c7900fbc6ffc None None] MFA Rules not processed for user `305d1ce79a4f43c8b98e0967dab43da1`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:17:18.842973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-d2981c09-4571-4f9b-9b9c-c7900fbc6ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:17:18.850898 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1377/2766] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:17:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:17:21.595743 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=68 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:17:28.837026 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:18:21.291992 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c3d28-f146-432d-bc16-a03200e7ce6a service cinder] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:18:21.294555 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-b77c3d28-f146-432d-bc16-a03200e7ce6a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:18:21.380717 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c3d28-f146-432d-bc16-a03200e7ce6a service cinder] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:18:21.385047 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c947f-c8d4-477b-b285-9a79dfbafce8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:18:21.388625 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c947f-c8d4-477b-b285-9a79dfbafce8 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:18:21.388625 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c947f-c8d4-477b-b285-9a79dfbafce8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:18:21.388625 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c947f-c8d4-477b-b285-9a79dfbafce8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:18:21.388625 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-3c0c947f-c8d4-477b-b285-9a79dfbafce8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:18:21.400414 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0c947f-c8d4-477b-b285-9a79dfbafce8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:18:21.402223 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0c947f-c8d4-477b-b285-9a79dfbafce8 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:18:21.412641 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1390/2767] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:18:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:18:21.452774 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2981c09-4571-4f9b-9b9c-c7900fbc6ffc None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:18:21.469328 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2981c09-4571-4f9b-9b9c-c7900fbc6ffc None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:18:21.476174 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece2040-1d8e-4aab-a137-e42554e6ed92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:18:21.477276 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece2040-1d8e-4aab-a137-e42554e6ed92 service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:18:21.477790 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece2040-1d8e-4aab-a137-e42554e6ed92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:18:21.478317 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece2040-1d8e-4aab-a137-e42554e6ed92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:18:21.481597 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-cece2040-1d8e-4aab-a137-e42554e6ed92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:18:21.574656 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece2040-1d8e-4aab-a137-e42554e6ed92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:18:21.576107 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece2040-1d8e-4aab-a137-e42554e6ed92 service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:18:21.592428 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1378/2768] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:18:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:18:22.809949 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c947f-c8d4-477b-b285-9a79dfbafce8 service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:18:22.822124 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c947f-c8d4-477b-b285-9a79dfbafce8 service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:18:22.826209 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a12c5-3ebc-4ea8-8058-6737dbfedc5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:18:22.826991 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a12c5-3ebc-4ea8-8058-6737dbfedc5e service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:18:22.828035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a12c5-3ebc-4ea8-8058-6737dbfedc5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:18:22.828619 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a12c5-3ebc-4ea8-8058-6737dbfedc5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:18:22.830705 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-749a12c5-3ebc-4ea8-8058-6737dbfedc5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:18:22.845991 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749a12c5-3ebc-4ea8-8058-6737dbfedc5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:18:22.847438 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749a12c5-3ebc-4ea8-8058-6737dbfedc5e service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:18:22.861914 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1391/2769] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:18:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:18:23.963014 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece2040-1d8e-4aab-a137-e42554e6ed92 service neutron] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:18:23.964671 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-cece2040-1d8e-4aab-a137-e42554e6ed92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:18:23.981451 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece2040-1d8e-4aab-a137-e42554e6ed92 service neutron] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:18:23.987966 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79655cb-3b5e-4828-87d4-49235a27f29c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:18:23.989082 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79655cb-3b5e-4828-87d4-49235a27f29c service nova] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:18:23.989082 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79655cb-3b5e-4828-87d4-49235a27f29c service nova] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:18:23.989392 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79655cb-3b5e-4828-87d4-49235a27f29c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:18:23.991653 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-c79655cb-3b5e-4828-87d4-49235a27f29c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:18:24.008889 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79655cb-3b5e-4828-87d4-49235a27f29c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:18:24.010686 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79655cb-3b5e-4828-87d4-49235a27f29c service nova] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:18:24.028093 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1379/2770] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:18:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:18:32.855688 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=32 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:18:33.455275 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a12c5-3ebc-4ea8-8058-6737dbfedc5e service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:18:33.464554 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a12c5-3ebc-4ea8-8058-6737dbfedc5e service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:18:33.471964 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d38e2-0d37-4239-abdc-8ba4ec512543 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:18:33.472869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d38e2-0d37-4239-abdc-8ba4ec512543 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:18:33.473378 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d38e2-0d37-4239-abdc-8ba4ec512543 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:18:33.473751 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d38e2-0d37-4239-abdc-8ba4ec512543 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:18:33.476013 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-c58d38e2-0d37-4239-abdc-8ba4ec512543 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:18:33.548476 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58d38e2-0d37-4239-abdc-8ba4ec512543 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:18:33.550081 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58d38e2-0d37-4239-abdc-8ba4ec512543 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:18:33.560739 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1392/2771] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:18:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:18:34.020232 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=40 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:18:40.556721 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79655cb-3b5e-4828-87d4-49235a27f29c service nova] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:18:40.558203 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-c79655cb-3b5e-4828-87d4-49235a27f29c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:18:40.626028 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d38e2-0d37-4239-abdc-8ba4ec512543 service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:18:40.629715 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-c58d38e2-0d37-4239-abdc-8ba4ec512543 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:18:40.652488 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d38e2-0d37-4239-abdc-8ba4ec512543 service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:18:40.658677 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79655cb-3b5e-4828-87d4-49235a27f29c service nova] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:18:40.660685 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b77a0-272f-4eef-a920-85b470b35716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:18:40.661639 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b77a0-272f-4eef-a920-85b470b35716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:18:40.661967 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b77a0-272f-4eef-a920-85b470b35716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:18:40.662422 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b77a0-272f-4eef-a920-85b470b35716 admin admin] PATH_INFO: `/v3/users/b5d959f3af3745ed8e0bc4a67faf34f1` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:18:40.664080 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b77a0-272f-4eef-a920-85b470b35716 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5d959f3af3745ed8e0bc4a67faf34f1)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:18:40.664549 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62066355-0df2-4b01-852a-606e2332fa43 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61e2c954f9a49abad9f3de17ee39f63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:18:40.665211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62066355-0df2-4b01-852a-606e2332fa43 service placement] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:18:40.665485 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62066355-0df2-4b01-852a-606e2332fa43 service placement] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:18:40.665804 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62066355-0df2-4b01-852a-606e2332fa43 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:18:40.666072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b77a0-272f-4eef-a920-85b470b35716 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:18:40.673730 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-62066355-0df2-4b01-852a-606e2332fa43 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:18:40.688212 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62066355-0df2-4b01-852a-606e2332fa43 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:18:40.689859 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62066355-0df2-4b01-852a-606e2332fa43 service placement] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:18:40.703372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1380/2772] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:18:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:18:40.720882 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-4f8b77a0-272f-4eef-a920-85b470b35716 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5d959f3af3745ed8e0bc4a67faf34f1'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:18:40.731082 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-4f8b77a0-272f-4eef-a920-85b470b35716 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5d959f3af3745ed8e0bc4a67faf34f1'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:18:40.740498 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-4f8b77a0-272f-4eef-a920-85b470b35716 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5d959f3af3745ed8e0bc4a67faf34f1'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:18:40.754183 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-4f8b77a0-272f-4eef-a920-85b470b35716 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5d959f3af3745ed8e0bc4a67faf34f1'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:18:40.754919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-4f8b77a0-272f-4eef-a920-85b470b35716 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5d959f3af3745ed8e0bc4a67faf34f1'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:18:40.761114 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1393/2773] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:18:40 2022] DELETE /identity/v3/users/b5d959f3af3745ed8e0bc4a67faf34f1 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:18:40.951383 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62066355-0df2-4b01-852a-606e2332fa43 service placement] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:18:40.965542 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62066355-0df2-4b01-852a-606e2332fa43 service placement] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:18:41.027675 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4060e3a7-0bc8-48b5-a6ff-0c48c720b688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:18:41.028317 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4060e3a7-0bc8-48b5-a6ff-0c48c720b688 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:18:41.028621 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4060e3a7-0bc8-48b5-a6ff-0c48c720b688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:18:41.028898 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4060e3a7-0bc8-48b5-a6ff-0c48c720b688 admin admin] PATH_INFO: `/v3/projects/cd1707d3a6c640b78baeb522cfb7ecdb` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:18:41.030091 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4060e3a7-0bc8-48b5-a6ff-0c48c720b688 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd1707d3a6c640b78baeb522cfb7ecdb)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:18:41.033431 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4060e3a7-0bc8-48b5-a6ff-0c48c720b688 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:18:41.086317 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-4060e3a7-0bc8-48b5-a6ff-0c48c720b688 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd1707d3a6c640b78baeb522cfb7ecdb', 'request_id': 'req-4060e3a7-0bc8-48b5-a6ff-0c48c720b688'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:18:41.094829 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-4060e3a7-0bc8-48b5-a6ff-0c48c720b688 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd1707d3a6c640b78baeb522cfb7ecdb', 'request_id': 'req-4060e3a7-0bc8-48b5-a6ff-0c48c720b688'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:18:41.100995 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-4060e3a7-0bc8-48b5-a6ff-0c48c720b688 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd1707d3a6c640b78baeb522cfb7ecdb', 'request_id': 'req-4060e3a7-0bc8-48b5-a6ff-0c48c720b688'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:18:41.113474 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-4060e3a7-0bc8-48b5-a6ff-0c48c720b688 admin admin] The token cache is being invalidate because project cd1707d3a6c640b78baeb522cfb7ecdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:18:41.113793 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-4060e3a7-0bc8-48b5-a6ff-0c48c720b688 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:18:41.114536 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-4060e3a7-0bc8-48b5-a6ff-0c48c720b688 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:18:41.116119 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1381/2774] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:18:40 2022] DELETE /identity/v3/projects/cd1707d3a6c640b78baeb522cfb7ecdb => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:18:41.120534 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b77a0-272f-4eef-a920-85b470b35716 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:18:41.123104 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-4f8b77a0-272f-4eef-a920-85b470b35716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:18:41.136768 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b77a0-272f-4eef-a920-85b470b35716 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:18:41.198909 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497398f4-bece-4864-8ca2-5c1efb64fdbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:18:41.199552 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497398f4-bece-4864-8ca2-5c1efb64fdbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:18:41.199836 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497398f4-bece-4864-8ca2-5c1efb64fdbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:18:41.200111 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497398f4-bece-4864-8ca2-5c1efb64fdbe admin admin] PATH_INFO: `/v3/users/901fcc9fa690437f9cc2a30f6f4204ac` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:18:41.201302 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497398f4-bece-4864-8ca2-5c1efb64fdbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=901fcc9fa690437f9cc2a30f6f4204ac)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:18:41.204534 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497398f4-bece-4864-8ca2-5c1efb64fdbe admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:18:41.210032 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:18:41.257946 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-497398f4-bece-4864-8ca2-5c1efb64fdbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '901fcc9fa690437f9cc2a30f6f4204ac'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:18:41.267508 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-497398f4-bece-4864-8ca2-5c1efb64fdbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '901fcc9fa690437f9cc2a30f6f4204ac'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:18:41.275143 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-497398f4-bece-4864-8ca2-5c1efb64fdbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '901fcc9fa690437f9cc2a30f6f4204ac'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:18:41.289031 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-497398f4-bece-4864-8ca2-5c1efb64fdbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '901fcc9fa690437f9cc2a30f6f4204ac'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:18:41.290180 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-497398f4-bece-4864-8ca2-5c1efb64fdbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '901fcc9fa690437f9cc2a30f6f4204ac'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:18:41.296408 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1394/2775] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:18:41 2022] DELETE /identity/v3/users/901fcc9fa690437f9cc2a30f6f4204ac => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:18:41.481447 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4060e3a7-0bc8-48b5-a6ff-0c48c720b688 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:18:41.483223 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-4060e3a7-0bc8-48b5-a6ff-0c48c720b688 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:18:41.498229 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4060e3a7-0bc8-48b5-a6ff-0c48c720b688 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:18:41.518122 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:18:41.563820 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59d657-65ca-4022-bc01-08e7f85526b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:18:41.564553 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59d657-65ca-4022-bc01-08e7f85526b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:18:41.564757 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59d657-65ca-4022-bc01-08e7f85526b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:18:41.565033 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59d657-65ca-4022-bc01-08e7f85526b2 admin admin] PATH_INFO: `/v3/projects/b70d4c679d90460996b6e0bdfa4ea50a` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:18:41.566201 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff59d657-65ca-4022-bc01-08e7f85526b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b70d4c679d90460996b6e0bdfa4ea50a)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:18:41.568091 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff59d657-65ca-4022-bc01-08e7f85526b2 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:18:41.636428 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ff59d657-65ca-4022-bc01-08e7f85526b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b70d4c679d90460996b6e0bdfa4ea50a', 'request_id': 'req-ff59d657-65ca-4022-bc01-08e7f85526b2'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:18:41.646420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ff59d657-65ca-4022-bc01-08e7f85526b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b70d4c679d90460996b6e0bdfa4ea50a', 'request_id': 'req-ff59d657-65ca-4022-bc01-08e7f85526b2'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:18:41.657849 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ff59d657-65ca-4022-bc01-08e7f85526b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b70d4c679d90460996b6e0bdfa4ea50a', 'request_id': 'req-ff59d657-65ca-4022-bc01-08e7f85526b2'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:18:41.670361 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ff59d657-65ca-4022-bc01-08e7f85526b2 admin admin] The token cache is being invalidate because project b70d4c679d90460996b6e0bdfa4ea50a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:18:41.670646 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ff59d657-65ca-4022-bc01-08e7f85526b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:18:41.671299 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ff59d657-65ca-4022-bc01-08e7f85526b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:18:41.673456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1382/2776] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:18:41 2022] DELETE /identity/v3/projects/b70d4c679d90460996b6e0bdfa4ea50a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:18:51.292887 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:18:51.667289 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:19:21.284969 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497398f4-bece-4864-8ca2-5c1efb64fdbe admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:21.286401 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-497398f4-bece-4864-8ca2-5c1efb64fdbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:21.348301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497398f4-bece-4864-8ca2-5c1efb64fdbe admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:21.400118 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa52b3-f070-4256-a610-969f5cec3a80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:21.400679 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa52b3-f070-4256-a610-969f5cec3a80 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:21.400991 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa52b3-f070-4256-a610-969f5cec3a80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:21.401270 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa52b3-f070-4256-a610-969f5cec3a80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:21.403038 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-69fa52b3-f070-4256-a610-969f5cec3a80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:21.478773 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa52b3-f070-4256-a610-969f5cec3a80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:21.480229 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa52b3-f070-4256-a610-969f5cec3a80 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:21.490127 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1395/2777] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:19:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:19:25.090311 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59d657-65ca-4022-bc01-08e7f85526b2 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:25.093565 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-ff59d657-65ca-4022-bc01-08e7f85526b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:25.114862 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59d657-65ca-4022-bc01-08e7f85526b2 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:25.121493 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3b17d-c9ae-43a7-a7c5-59c7de66e238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:25.122157 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3b17d-c9ae-43a7-a7c5-59c7de66e238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:25.122430 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3b17d-c9ae-43a7-a7c5-59c7de66e238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:25.122909 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3b17d-c9ae-43a7-a7c5-59c7de66e238 admin admin] PATH_INFO: `/v3/users/305d1ce79a4f43c8b98e0967dab43da1` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:25.125395 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b3b17d-c9ae-43a7-a7c5-59c7de66e238 admin admin] RBAC: Authorizing `identity:delete_user(user_id=305d1ce79a4f43c8b98e0967dab43da1)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:25.127769 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b3b17d-c9ae-43a7-a7c5-59c7de66e238 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:25.191508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-05b3b17d-c9ae-43a7-a7c5-59c7de66e238 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '305d1ce79a4f43c8b98e0967dab43da1'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:25.201178 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-05b3b17d-c9ae-43a7-a7c5-59c7de66e238 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '305d1ce79a4f43c8b98e0967dab43da1'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:25.228453 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-05b3b17d-c9ae-43a7-a7c5-59c7de66e238 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '305d1ce79a4f43c8b98e0967dab43da1'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:25.229261 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-05b3b17d-c9ae-43a7-a7c5-59c7de66e238 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '305d1ce79a4f43c8b98e0967dab43da1'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:25.239342 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-05b3b17d-c9ae-43a7-a7c5-59c7de66e238 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '305d1ce79a4f43c8b98e0967dab43da1'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:25.257773 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1383/2778] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:19:25 2022] DELETE /identity/v3/users/305d1ce79a4f43c8b98e0967dab43da1 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:19:25.468522 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa52b3-f070-4256-a610-969f5cec3a80 service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:25.482093 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa52b3-f070-4256-a610-969f5cec3a80 service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:25.561835 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978327ab-1a0e-42dd-a7f2-3b180aedecdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:25.562473 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978327ab-1a0e-42dd-a7f2-3b180aedecdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:25.562745 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978327ab-1a0e-42dd-a7f2-3b180aedecdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:25.563012 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978327ab-1a0e-42dd-a7f2-3b180aedecdf admin admin] PATH_INFO: `/v3/projects/790b5019b9114120a51718d99317f7a1` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:25.564168 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978327ab-1a0e-42dd-a7f2-3b180aedecdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=790b5019b9114120a51718d99317f7a1)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:25.566098 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978327ab-1a0e-42dd-a7f2-3b180aedecdf admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:25.575817 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:19:25.637263 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-978327ab-1a0e-42dd-a7f2-3b180aedecdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '790b5019b9114120a51718d99317f7a1', 'request_id': 'req-978327ab-1a0e-42dd-a7f2-3b180aedecdf'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:25.647685 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-978327ab-1a0e-42dd-a7f2-3b180aedecdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '790b5019b9114120a51718d99317f7a1', 'request_id': 'req-978327ab-1a0e-42dd-a7f2-3b180aedecdf'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:25.658889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-978327ab-1a0e-42dd-a7f2-3b180aedecdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '790b5019b9114120a51718d99317f7a1', 'request_id': 'req-978327ab-1a0e-42dd-a7f2-3b180aedecdf'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:25.674801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-978327ab-1a0e-42dd-a7f2-3b180aedecdf admin admin] The token cache is being invalidate because project 790b5019b9114120a51718d99317f7a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:19:25.675115 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-978327ab-1a0e-42dd-a7f2-3b180aedecdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:25.675947 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-978327ab-1a0e-42dd-a7f2-3b180aedecdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:25.681033 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1396/2779] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:19:25 2022] DELETE /identity/v3/projects/790b5019b9114120a51718d99317f7a1 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:19:25.686672 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e49f1c-f72c-4373-bb2d-c7dce2c52a90 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:25.686803 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e49f1c-f72c-4373-bb2d-c7dce2c52a90 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:25.687077 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e49f1c-f72c-4373-bb2d-c7dce2c52a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:25.708314 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-34e49f1c-f72c-4373-bb2d-c7dce2c52a90 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:19:25.709660 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-34e49f1c-f72c-4373-bb2d-c7dce2c52a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:25.782517 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1384/2780] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:19:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:19:25.791257 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c159a-e044-47c4-9eeb-e8337d9b077c None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:25.791533 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c159a-e044-47c4-9eeb-e8337d9b077c None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:25.791887 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c159a-e044-47c4-9eeb-e8337d9b077c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:25.813556 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-626c159a-e044-47c4-9eeb-e8337d9b077c None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:19:25.814784 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-626c159a-e044-47c4-9eeb-e8337d9b077c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:25.822780 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1397/2781] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:19:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:19:25.827349 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e49f1c-f72c-4373-bb2d-c7dce2c52a90 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:25.837295 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e49f1c-f72c-4373-bb2d-c7dce2c52a90 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:25.841929 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f5c72-65f5-4225-b814-45d96aa5c523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:25.843128 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f5c72-65f5-4225-b814-45d96aa5c523 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:25.843411 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f5c72-65f5-4225-b814-45d96aa5c523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:25.843679 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f5c72-65f5-4225-b814-45d96aa5c523 admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:25.844505 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9f5c72-65f5-4225-b814-45d96aa5c523 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:19:25.845139 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9f5c72-65f5-4225-b814-45d96aa5c523 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:25.846334 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9f5c72-65f5-4225-b814-45d96aa5c523 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:25.859271 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1385/2782] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:19:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:19:25.865011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c159a-e044-47c4-9eeb-e8337d9b077c None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:25.875269 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c159a-e044-47c4-9eeb-e8337d9b077c None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:25.882144 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7fc949-db7b-41f1-a4e9-64f07ba1ebe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:25.882760 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7fc949-db7b-41f1-a4e9-64f07ba1ebe3 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:25.883036 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7fc949-db7b-41f1-a4e9-64f07ba1ebe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:25.883332 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7fc949-db7b-41f1-a4e9-64f07ba1ebe3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:25.884034 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7fc949-db7b-41f1-a4e9-64f07ba1ebe3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:25.886002 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7fc949-db7b-41f1-a4e9-64f07ba1ebe3 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:25.916375 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1398/2783] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:19:25 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:19:25.922042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f5c72-65f5-4225-b814-45d96aa5c523 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:25.934530 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f5c72-65f5-4225-b814-45d96aa5c523 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:26.005233 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3d7d99-794c-4284-807e-07edf078d740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:26.005865 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3d7d99-794c-4284-807e-07edf078d740 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:26.006146 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3d7d99-794c-4284-807e-07edf078d740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:26.006425 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3d7d99-794c-4284-807e-07edf078d740 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:26.007119 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3d7d99-794c-4284-807e-07edf078d740 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:26.009119 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3d7d99-794c-4284-807e-07edf078d740 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:26.018106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:19:26.043468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1386/2784] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:19:25 2022] POST /identity/v3/users => generated 294 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:19:26.044979 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7fc949-db7b-41f1-a4e9-64f07ba1ebe3 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:26.056316 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7fc949-db7b-41f1-a4e9-64f07ba1ebe3 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:26.062434 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c76dfd-cc18-496c-baa3-ea1dfea78fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:26.063063 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c76dfd-cc18-496c-baa3-ea1dfea78fca admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:26.063346 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c76dfd-cc18-496c-baa3-ea1dfea78fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:26.063617 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c76dfd-cc18-496c-baa3-ea1dfea78fca admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:26.064321 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c76dfd-cc18-496c-baa3-ea1dfea78fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:26.066034 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c76dfd-cc18-496c-baa3-ea1dfea78fca admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:26.078387 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1399/2785] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:19:26.083816 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3d7d99-794c-4284-807e-07edf078d740 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:26.102039 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3d7d99-794c-4284-807e-07edf078d740 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:26.111068 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b4ee3-41c9-4161-8546-a2bb3e290ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:26.112702 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b4ee3-41c9-4161-8546-a2bb3e290ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:26.113204 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b4ee3-41c9-4161-8546-a2bb3e290ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:26.113681 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b4ee3-41c9-4161-8546-a2bb3e290ae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:26.116243 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006b4ee3-41c9-4161-8546-a2bb3e290ae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:26.118985 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006b4ee3-41c9-4161-8546-a2bb3e290ae5 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:26.132006 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1387/2786] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:19:26.136360 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c76dfd-cc18-496c-baa3-ea1dfea78fca admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:26.147611 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c76dfd-cc18-496c-baa3-ea1dfea78fca admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:26.152459 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea35f1-8a9c-4145-bf6d-8477dfd50d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:26.153186 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea35f1-8a9c-4145-bf6d-8477dfd50d16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:26.153468 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea35f1-8a9c-4145-bf6d-8477dfd50d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:26.153946 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea35f1-8a9c-4145-bf6d-8477dfd50d16 admin admin] PATH_INFO: `/v3/projects/cd4bcd3c700140aea318d1723fe57e35/users/0c6b4448abd848c2abca9d6e3b879e12/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:26.172239 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea35f1-8a9c-4145-bf6d-8477dfd50d16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd4bcd3c700140aea318d1723fe57e35, user_id=0c6b4448abd848c2abca9d6e3b879e12, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:26.174948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea35f1-8a9c-4145-bf6d-8477dfd50d16 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:26.190036 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1400/2787] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:19:26 2022] PUT /identity/v3/projects/cd4bcd3c700140aea318d1723fe57e35/users/0c6b4448abd848c2abca9d6e3b879e12/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:19:26.202907 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de648c-a93c-45d1-9162-4e50c968e984 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:26.203192 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de648c-a93c-45d1-9162-4e50c968e984 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:26.203447 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de648c-a93c-45d1-9162-4e50c968e984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:26.235498 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-c7de648c-a93c-45d1-9162-4e50c968e984 None None] MFA Rules not processed for user `0c6b4448abd848c2abca9d6e3b879e12`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:19:26.236749 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-c7de648c-a93c-45d1-9162-4e50c968e984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:26.306718 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1388/2788] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:19:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:19:26.311474 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea35f1-8a9c-4145-bf6d-8477dfd50d16 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:26.393223 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea35f1-8a9c-4145-bf6d-8477dfd50d16 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:26.398451 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bceae3-a604-4682-8afd-0d1f55872e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:26.399086 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bceae3-a604-4682-8afd-0d1f55872e4d admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:26.399373 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bceae3-a604-4682-8afd-0d1f55872e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:26.399650 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bceae3-a604-4682-8afd-0d1f55872e4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:26.400324 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bceae3-a604-4682-8afd-0d1f55872e4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:26.402301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bceae3-a604-4682-8afd-0d1f55872e4d admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:26.434442 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1401/2789] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:19:26 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:19:26.439389 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de648c-a93c-45d1-9162-4e50c968e984 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:26.452450 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de648c-a93c-45d1-9162-4e50c968e984 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:26.531423 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d5453f-c352-4407-8a3b-ad07a3bbc836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:26.532061 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5453f-c352-4407-8a3b-ad07a3bbc836 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:26.532224 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5453f-c352-4407-8a3b-ad07a3bbc836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:26.532477 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5453f-c352-4407-8a3b-ad07a3bbc836 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:26.533187 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d5453f-c352-4407-8a3b-ad07a3bbc836 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:26.535072 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d5453f-c352-4407-8a3b-ad07a3bbc836 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:26.558621 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1389/2790] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:19:26 2022] POST /identity/v3/users => generated 294 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:19:26.565712 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bceae3-a604-4682-8afd-0d1f55872e4d admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:26.577840 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bceae3-a604-4682-8afd-0d1f55872e4d admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:26.583856 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1a0b7-8671-4c85-a747-dcab213608ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:26.583856 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1a0b7-8671-4c85-a747-dcab213608ba admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:26.583856 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1a0b7-8671-4c85-a747-dcab213608ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:26.583856 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1a0b7-8671-4c85-a747-dcab213608ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:26.584566 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1a0b7-8671-4c85-a747-dcab213608ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:26.586172 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1a0b7-8671-4c85-a747-dcab213608ba admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:26.597829 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1402/2791] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:19:26.602155 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d5453f-c352-4407-8a3b-ad07a3bbc836 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:26.613012 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d5453f-c352-4407-8a3b-ad07a3bbc836 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:26.618022 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e6caf-3360-4b73-be20-ce04f0fc9a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:26.618682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e6caf-3360-4b73-be20-ce04f0fc9a39 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:26.619008 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e6caf-3360-4b73-be20-ce04f0fc9a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:26.619303 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e6caf-3360-4b73-be20-ce04f0fc9a39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:26.620173 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e6caf-3360-4b73-be20-ce04f0fc9a39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:26.621826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e6caf-3360-4b73-be20-ce04f0fc9a39 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:26.633565 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1390/2792] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:19:26.637891 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1a0b7-8671-4c85-a747-dcab213608ba admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:26.650932 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1a0b7-8671-4c85-a747-dcab213608ba admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:26.655777 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e008f4-37ec-405a-a66f-4cd7e9fdf215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:26.656437 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e008f4-37ec-405a-a66f-4cd7e9fdf215 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:26.656869 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e008f4-37ec-405a-a66f-4cd7e9fdf215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:26.657142 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e008f4-37ec-405a-a66f-4cd7e9fdf215 admin admin] PATH_INFO: `/v3/projects/887ecc26a8204e6cb47900e12b08914f/users/2fd29fbe5f424f9182e723620798b02e/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:26.659525 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:19:26.675296 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e008f4-37ec-405a-a66f-4cd7e9fdf215 admin admin] RBAC: Authorizing `identity:create_grant(project_id=887ecc26a8204e6cb47900e12b08914f, user_id=2fd29fbe5f424f9182e723620798b02e, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:26.678035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e008f4-37ec-405a-a66f-4cd7e9fdf215 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:26.695211 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1403/2793] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:19:26 2022] PUT /identity/v3/projects/887ecc26a8204e6cb47900e12b08914f/users/2fd29fbe5f424f9182e723620798b02e/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:19:26.712490 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb613aa-0af4-4c38-9e0c-e2be5c992c45 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:26.712811 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb613aa-0af4-4c38-9e0c-e2be5c992c45 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:26.713067 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb613aa-0af4-4c38-9e0c-e2be5c992c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:26.743702 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-5cb613aa-0af4-4c38-9e0c-e2be5c992c45 None None] MFA Rules not processed for user `2fd29fbe5f424f9182e723620798b02e`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:19:26.744955 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-5cb613aa-0af4-4c38-9e0c-e2be5c992c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:26.780663 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:19:26.825206 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1391/2794] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:19:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:19:26.830167 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e008f4-37ec-405a-a66f-4cd7e9fdf215 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:26.896350 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e008f4-37ec-405a-a66f-4cd7e9fdf215 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:26.901764 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b183f-3326-40f0-8e1b-8215c1f47299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:26.902382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b183f-3326-40f0-8e1b-8215c1f47299 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:26.902662 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b183f-3326-40f0-8e1b-8215c1f47299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:26.902932 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b183f-3326-40f0-8e1b-8215c1f47299 admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:26.903601 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4b183f-3326-40f0-8e1b-8215c1f47299 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:26.906531 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4b183f-3326-40f0-8e1b-8215c1f47299 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:26.935260 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1404/2795] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:19:26 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:19:26.940104 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb613aa-0af4-4c38-9e0c-e2be5c992c45 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:26.952220 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb613aa-0af4-4c38-9e0c-e2be5c992c45 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:27.011327 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80eb34-c128-462f-9363-28cf68d7f9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:27.011952 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80eb34-c128-462f-9363-28cf68d7f9f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:27.012234 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80eb34-c128-462f-9363-28cf68d7f9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:27.012508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80eb34-c128-462f-9363-28cf68d7f9f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:27.013224 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c80eb34-c128-462f-9363-28cf68d7f9f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:27.016139 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c80eb34-c128-462f-9363-28cf68d7f9f5 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:27.041267 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1392/2796] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:19:26 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:19:27.046408 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b183f-3326-40f0-8e1b-8215c1f47299 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:27.058874 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b183f-3326-40f0-8e1b-8215c1f47299 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:27.063937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2edd2-0bb2-49ff-b736-ac3aa2eff70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:27.064545 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2edd2-0bb2-49ff-b736-ac3aa2eff70f admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:27.064871 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2edd2-0bb2-49ff-b736-ac3aa2eff70f admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:27.065146 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2edd2-0bb2-49ff-b736-ac3aa2eff70f admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:27.065989 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a2edd2-0bb2-49ff-b736-ac3aa2eff70f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:27.067479 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a2edd2-0bb2-49ff-b736-ac3aa2eff70f admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:27.082153 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1405/2797] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:19:27.086543 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80eb34-c128-462f-9363-28cf68d7f9f5 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:27.098800 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80eb34-c128-462f-9363-28cf68d7f9f5 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:27.103482 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e35d42-49b2-4bda-ab34-ef07aa2239da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:27.104160 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e35d42-49b2-4bda-ab34-ef07aa2239da admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:27.104444 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e35d42-49b2-4bda-ab34-ef07aa2239da admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:27.104758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e35d42-49b2-4bda-ab34-ef07aa2239da admin admin] PATH_INFO: `/v3/projects/6ae0fa00a1174b4ab501ed271bd1f6e8/users/793ff980e60446119418c5871f5bb149/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:27.122572 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e35d42-49b2-4bda-ab34-ef07aa2239da admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ae0fa00a1174b4ab501ed271bd1f6e8, user_id=793ff980e60446119418c5871f5bb149, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:27.125405 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e35d42-49b2-4bda-ab34-ef07aa2239da admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:27.140147 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1393/2798] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:19:27 2022] PUT /identity/v3/projects/6ae0fa00a1174b4ab501ed271bd1f6e8/users/793ff980e60446119418c5871f5bb149/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:19:27.148829 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2edd2-0bb2-49ff-b736-ac3aa2eff70f admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:27.160830 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2edd2-0bb2-49ff-b736-ac3aa2eff70f admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:27.221050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f8132f-c58e-4144-8683-31d188576ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:27.221676 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8132f-c58e-4144-8683-31d188576ed8 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:27.221952 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8132f-c58e-4144-8683-31d188576ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:27.222249 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8132f-c58e-4144-8683-31d188576ed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:27.222945 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f8132f-c58e-4144-8683-31d188576ed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:27.224537 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f8132f-c58e-4144-8683-31d188576ed8 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:27.235086 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1406/2799] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:19:27.239703 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e35d42-49b2-4bda-ab34-ef07aa2239da admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:27.251044 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e35d42-49b2-4bda-ab34-ef07aa2239da admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:27.256284 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974701f3-5a87-4f28-ac74-4d6d997338a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:27.257022 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974701f3-5a87-4f28-ac74-4d6d997338a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:27.257298 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974701f3-5a87-4f28-ac74-4d6d997338a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:27.257572 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974701f3-5a87-4f28-ac74-4d6d997338a3 admin admin] PATH_INFO: `/v3/projects/6ae0fa00a1174b4ab501ed271bd1f6e8/users/793ff980e60446119418c5871f5bb149/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:27.259655 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974701f3-5a87-4f28-ac74-4d6d997338a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ae0fa00a1174b4ab501ed271bd1f6e8, user_id=793ff980e60446119418c5871f5bb149, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:27.262372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974701f3-5a87-4f28-ac74-4d6d997338a3 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:27.275010 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1394/2800] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:19:27 2022] PUT /identity/v3/projects/6ae0fa00a1174b4ab501ed271bd1f6e8/users/793ff980e60446119418c5871f5bb149/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:19:27.286291 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aabddfa-02ab-4eee-970f-dd11241d96c6 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:27.286578 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aabddfa-02ab-4eee-970f-dd11241d96c6 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:27.286854 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aabddfa-02ab-4eee-970f-dd11241d96c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:27.319511 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-6aabddfa-02ab-4eee-970f-dd11241d96c6 None None] MFA Rules not processed for user `793ff980e60446119418c5871f5bb149`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:19:27.320702 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-6aabddfa-02ab-4eee-970f-dd11241d96c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:27.368648 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:19:27.401944 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1407/2801] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:19:27 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:19:27.418479 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974701f3-5a87-4f28-ac74-4d6d997338a3 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:27.420267 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-974701f3-5a87-4f28-ac74-4d6d997338a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:27.484272 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974701f3-5a87-4f28-ac74-4d6d997338a3 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:27.488029 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee5a4a-7487-4fb5-91ba-bb8fe02e0364 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:27.488680 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee5a4a-7487-4fb5-91ba-bb8fe02e0364 service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:27.488961 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee5a4a-7487-4fb5-91ba-bb8fe02e0364 service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:27.489234 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee5a4a-7487-4fb5-91ba-bb8fe02e0364 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:27.499548 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee5a4a-7487-4fb5-91ba-bb8fe02e0364 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:27.501025 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee5a4a-7487-4fb5-91ba-bb8fe02e0364 service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:27.511136 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1395/2802] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:19:27 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:19:28.034096 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aabddfa-02ab-4eee-970f-dd11241d96c6 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:28.035530 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-6aabddfa-02ab-4eee-970f-dd11241d96c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:28.049395 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aabddfa-02ab-4eee-970f-dd11241d96c6 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:28.053136 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b7c18-ccc7-43ab-b5af-e9bd0c9623c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:28.053744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b7c18-ccc7-43ab-b5af-e9bd0c9623c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:28.054021 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b7c18-ccc7-43ab-b5af-e9bd0c9623c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:28.054317 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b7c18-ccc7-43ab-b5af-e9bd0c9623c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:28.115861 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b7c18-ccc7-43ab-b5af-e9bd0c9623c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:28.117345 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b7c18-ccc7-43ab-b5af-e9bd0c9623c2 service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:28.126829 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1408/2803] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:19:28 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:19:32.212452 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee5a4a-7487-4fb5-91ba-bb8fe02e0364 service cinder] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:32.223731 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee5a4a-7487-4fb5-91ba-bb8fe02e0364 service cinder] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:32.227498 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b37e087-41ed-4f43-a1e0-81265ac49ec8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:32.228123 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37e087-41ed-4f43-a1e0-81265ac49ec8 service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:32.228404 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37e087-41ed-4f43-a1e0-81265ac49ec8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:32.228705 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37e087-41ed-4f43-a1e0-81265ac49ec8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:32.267419 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:19:32.299681 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b37e087-41ed-4f43-a1e0-81265ac49ec8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:32.301179 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b37e087-41ed-4f43-a1e0-81265ac49ec8 service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:32.317433 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1396/2804] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:19:32 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:19:34.680759 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b7c18-ccc7-43ab-b5af-e9bd0c9623c2 service cinder] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:34.758380 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b7c18-ccc7-43ab-b5af-e9bd0c9623c2 service cinder] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:34.763289 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8c26c-818e-44c9-a00b-945f3854ccdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:34.764983 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8c26c-818e-44c9-a00b-945f3854ccdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:34.765472 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8c26c-818e-44c9-a00b-945f3854ccdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:34.765969 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8c26c-818e-44c9-a00b-945f3854ccdc admin admin] PATH_INFO: `/v3/users/0c6b4448abd848c2abca9d6e3b879e12` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:34.767583 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d8c26c-818e-44c9-a00b-945f3854ccdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c6b4448abd848c2abca9d6e3b879e12)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:34.769992 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d8c26c-818e-44c9-a00b-945f3854ccdc admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:34.825063 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-55d8c26c-818e-44c9-a00b-945f3854ccdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c6b4448abd848c2abca9d6e3b879e12'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:34.834752 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-55d8c26c-818e-44c9-a00b-945f3854ccdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c6b4448abd848c2abca9d6e3b879e12'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:34.842744 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-55d8c26c-818e-44c9-a00b-945f3854ccdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c6b4448abd848c2abca9d6e3b879e12'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:34.854826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:19:34.856050 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-55d8c26c-818e-44c9-a00b-945f3854ccdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c6b4448abd848c2abca9d6e3b879e12'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:34.856900 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-55d8c26c-818e-44c9-a00b-945f3854ccdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c6b4448abd848c2abca9d6e3b879e12'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:34.867113 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1409/2805] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:19:34 2022] DELETE /identity/v3/users/0c6b4448abd848c2abca9d6e3b879e12 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:19:34.872080 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fa3fc-eb5b-4eda-99fd-4e04e874315f None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:34.872992 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fa3fc-eb5b-4eda-99fd-4e04e874315f None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:34.872992 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fa3fc-eb5b-4eda-99fd-4e04e874315f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:34.894198 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-444fa3fc-eb5b-4eda-99fd-4e04e874315f None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:19:34.895438 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-444fa3fc-eb5b-4eda-99fd-4e04e874315f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:34.959149 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1397/2806] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:19:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:19:34.969003 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8c26c-818e-44c9-a00b-945f3854ccdc admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:34.970565 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-55d8c26c-818e-44c9-a00b-945f3854ccdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:35.037052 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8c26c-818e-44c9-a00b-945f3854ccdc admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:35.041167 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1047d-5dc1-4d81-b076-12caa6b516e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:35.041792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1047d-5dc1-4d81-b076-12caa6b516e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:35.042073 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1047d-5dc1-4d81-b076-12caa6b516e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:35.042346 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1047d-5dc1-4d81-b076-12caa6b516e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:35.053210 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1047d-5dc1-4d81-b076-12caa6b516e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:35.054526 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1047d-5dc1-4d81-b076-12caa6b516e9 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:35.066026 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1410/2807] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:19:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:19:35.383926 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444fa3fc-eb5b-4eda-99fd-4e04e874315f None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:35.385383 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-444fa3fc-eb5b-4eda-99fd-4e04e874315f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:35.398339 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444fa3fc-eb5b-4eda-99fd-4e04e874315f None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:35.403092 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96457df9-349e-4e33-9bf1-37a571b5f8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:35.403733 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96457df9-349e-4e33-9bf1-37a571b5f8b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:35.404009 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96457df9-349e-4e33-9bf1-37a571b5f8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:35.404342 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96457df9-349e-4e33-9bf1-37a571b5f8b6 admin admin] PATH_INFO: `/v3/projects/cd4bcd3c700140aea318d1723fe57e35` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:35.406578 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96457df9-349e-4e33-9bf1-37a571b5f8b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd4bcd3c700140aea318d1723fe57e35)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:35.408506 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96457df9-349e-4e33-9bf1-37a571b5f8b6 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:35.458366 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-96457df9-349e-4e33-9bf1-37a571b5f8b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd4bcd3c700140aea318d1723fe57e35', 'request_id': 'req-96457df9-349e-4e33-9bf1-37a571b5f8b6'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:35.466209 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-96457df9-349e-4e33-9bf1-37a571b5f8b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd4bcd3c700140aea318d1723fe57e35', 'request_id': 'req-96457df9-349e-4e33-9bf1-37a571b5f8b6'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:35.473277 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-96457df9-349e-4e33-9bf1-37a571b5f8b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd4bcd3c700140aea318d1723fe57e35', 'request_id': 'req-96457df9-349e-4e33-9bf1-37a571b5f8b6'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:35.485189 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-96457df9-349e-4e33-9bf1-37a571b5f8b6 admin admin] The token cache is being invalidate because project cd4bcd3c700140aea318d1723fe57e35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:19:35.485502 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-96457df9-349e-4e33-9bf1-37a571b5f8b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:35.486211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-96457df9-349e-4e33-9bf1-37a571b5f8b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:35.487683 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1398/2808] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:19:35 2022] DELETE /identity/v3/projects/cd4bcd3c700140aea318d1723fe57e35 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:19:35.491854 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1047d-5dc1-4d81-b076-12caa6b516e9 service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:35.501855 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1047d-5dc1-4d81-b076-12caa6b516e9 service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:35.558986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2575e96-d4f3-4683-9600-b9c6a74a39d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:35.559524 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2575e96-d4f3-4683-9600-b9c6a74a39d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:35.559800 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2575e96-d4f3-4683-9600-b9c6a74a39d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:35.560069 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2575e96-d4f3-4683-9600-b9c6a74a39d6 admin admin] PATH_INFO: `/v3/users/2fd29fbe5f424f9182e723620798b02e` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:35.561916 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2575e96-d4f3-4683-9600-b9c6a74a39d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fd29fbe5f424f9182e723620798b02e)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:35.563807 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2575e96-d4f3-4683-9600-b9c6a74a39d6 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:35.611212 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-f2575e96-d4f3-4683-9600-b9c6a74a39d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fd29fbe5f424f9182e723620798b02e'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:35.618990 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-f2575e96-d4f3-4683-9600-b9c6a74a39d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fd29fbe5f424f9182e723620798b02e'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:35.626978 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-f2575e96-d4f3-4683-9600-b9c6a74a39d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fd29fbe5f424f9182e723620798b02e'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:35.640240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-f2575e96-d4f3-4683-9600-b9c6a74a39d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fd29fbe5f424f9182e723620798b02e'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:35.641082 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-f2575e96-d4f3-4683-9600-b9c6a74a39d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fd29fbe5f424f9182e723620798b02e'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:35.648256 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1411/2809] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:19:35 2022] DELETE /identity/v3/users/2fd29fbe5f424f9182e723620798b02e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:19:35.945760 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96457df9-349e-4e33-9bf1-37a571b5f8b6 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:35.948671 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-96457df9-349e-4e33-9bf1-37a571b5f8b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:35.969722 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96457df9-349e-4e33-9bf1-37a571b5f8b6 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:36.028750 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:19:36.042808 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99276c9b-4bb8-4569-adeb-b340d314e7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:36.043473 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99276c9b-4bb8-4569-adeb-b340d314e7fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:36.043768 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99276c9b-4bb8-4569-adeb-b340d314e7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:36.044211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99276c9b-4bb8-4569-adeb-b340d314e7fa admin admin] PATH_INFO: `/v3/projects/887ecc26a8204e6cb47900e12b08914f` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:36.046528 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99276c9b-4bb8-4569-adeb-b340d314e7fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=887ecc26a8204e6cb47900e12b08914f)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:36.048795 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99276c9b-4bb8-4569-adeb-b340d314e7fa admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:36.103511 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-99276c9b-4bb8-4569-adeb-b340d314e7fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '887ecc26a8204e6cb47900e12b08914f', 'request_id': 'req-99276c9b-4bb8-4569-adeb-b340d314e7fa'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:36.118178 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-99276c9b-4bb8-4569-adeb-b340d314e7fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '887ecc26a8204e6cb47900e12b08914f', 'request_id': 'req-99276c9b-4bb8-4569-adeb-b340d314e7fa'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:36.126678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-99276c9b-4bb8-4569-adeb-b340d314e7fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '887ecc26a8204e6cb47900e12b08914f', 'request_id': 'req-99276c9b-4bb8-4569-adeb-b340d314e7fa'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:36.154862 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-99276c9b-4bb8-4569-adeb-b340d314e7fa admin admin] The token cache is being invalidate because project 887ecc26a8204e6cb47900e12b08914f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:19:36.155291 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-99276c9b-4bb8-4569-adeb-b340d314e7fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:36.156883 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-99276c9b-4bb8-4569-adeb-b340d314e7fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:36.159093 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1399/2810] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:19:35 2022] DELETE /identity/v3/projects/887ecc26a8204e6cb47900e12b08914f => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:19:36.163570 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2575e96-d4f3-4683-9600-b9c6a74a39d6 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:36.165888 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-f2575e96-d4f3-4683-9600-b9c6a74a39d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:36.182067 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2575e96-d4f3-4683-9600-b9c6a74a39d6 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:36.196225 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:19:36.259828 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74af5c-5666-4402-b4a0-6e376b0e7083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:36.260490 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74af5c-5666-4402-b4a0-6e376b0e7083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:36.260801 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74af5c-5666-4402-b4a0-6e376b0e7083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:36.261078 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74af5c-5666-4402-b4a0-6e376b0e7083 admin admin] PATH_INFO: `/v3/users/793ff980e60446119418c5871f5bb149` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:36.262298 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f74af5c-5666-4402-b4a0-6e376b0e7083 admin admin] RBAC: Authorizing `identity:delete_user(user_id=793ff980e60446119418c5871f5bb149)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:36.264248 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f74af5c-5666-4402-b4a0-6e376b0e7083 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:36.335960 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-8f74af5c-5666-4402-b4a0-6e376b0e7083 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '793ff980e60446119418c5871f5bb149'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:36.345510 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-8f74af5c-5666-4402-b4a0-6e376b0e7083 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '793ff980e60446119418c5871f5bb149'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:36.353687 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-8f74af5c-5666-4402-b4a0-6e376b0e7083 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '793ff980e60446119418c5871f5bb149'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:36.365856 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-8f74af5c-5666-4402-b4a0-6e376b0e7083 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '793ff980e60446119418c5871f5bb149'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:36.366586 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-8f74af5c-5666-4402-b4a0-6e376b0e7083 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '793ff980e60446119418c5871f5bb149'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:36.372910 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1412/2811] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:19:36 2022] DELETE /identity/v3/users/793ff980e60446119418c5871f5bb149 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:19:36.678198 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99276c9b-4bb8-4569-adeb-b340d314e7fa admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:36.680230 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-99276c9b-4bb8-4569-adeb-b340d314e7fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:36.703020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99276c9b-4bb8-4569-adeb-b340d314e7fa admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:36.794529 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c22f2e-2fab-4f2b-affe-cbd4ecf6de47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:36.795393 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c22f2e-2fab-4f2b-affe-cbd4ecf6de47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:36.795863 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c22f2e-2fab-4f2b-affe-cbd4ecf6de47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:36.796292 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c22f2e-2fab-4f2b-affe-cbd4ecf6de47 admin admin] PATH_INFO: `/v3/projects/6ae0fa00a1174b4ab501ed271bd1f6e8` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:36.797792 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c22f2e-2fab-4f2b-affe-cbd4ecf6de47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ae0fa00a1174b4ab501ed271bd1f6e8)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:36.799855 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c22f2e-2fab-4f2b-affe-cbd4ecf6de47 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:36.853563 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-67c22f2e-2fab-4f2b-affe-cbd4ecf6de47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ae0fa00a1174b4ab501ed271bd1f6e8', 'request_id': 'req-67c22f2e-2fab-4f2b-affe-cbd4ecf6de47'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:36.862535 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-67c22f2e-2fab-4f2b-affe-cbd4ecf6de47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ae0fa00a1174b4ab501ed271bd1f6e8', 'request_id': 'req-67c22f2e-2fab-4f2b-affe-cbd4ecf6de47'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:36.868803 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-67c22f2e-2fab-4f2b-affe-cbd4ecf6de47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ae0fa00a1174b4ab501ed271bd1f6e8', 'request_id': 'req-67c22f2e-2fab-4f2b-affe-cbd4ecf6de47'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:36.880485 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-67c22f2e-2fab-4f2b-affe-cbd4ecf6de47 admin admin] The token cache is being invalidate because project 6ae0fa00a1174b4ab501ed271bd1f6e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:19:36.880799 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-67c22f2e-2fab-4f2b-affe-cbd4ecf6de47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:36.881438 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-67c22f2e-2fab-4f2b-affe-cbd4ecf6de47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:36.882949 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1400/2812] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:19:36 2022] DELETE /identity/v3/projects/6ae0fa00a1174b4ab501ed271bd1f6e8 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:19:36.890354 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252760d3-7f17-4146-80cd-780ee050b283 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:36.890647 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252760d3-7f17-4146-80cd-780ee050b283 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:36.890923 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252760d3-7f17-4146-80cd-780ee050b283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:36.914937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-252760d3-7f17-4146-80cd-780ee050b283 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:19:36.916108 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-252760d3-7f17-4146-80cd-780ee050b283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:36.977569 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1413/2813] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:19:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:19:36.986671 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa19e7f-b12f-4f4b-8b97-454337bf5372 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:36.987219 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa19e7f-b12f-4f4b-8b97-454337bf5372 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:36.987344 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa19e7f-b12f-4f4b-8b97-454337bf5372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:37.007939 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-4aa19e7f-b12f-4f4b-8b97-454337bf5372 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:19:37.009260 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-4aa19e7f-b12f-4f4b-8b97-454337bf5372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:37.018036 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1401/2814] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:19:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:19:37.022696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252760d3-7f17-4146-80cd-780ee050b283 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:37.033335 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252760d3-7f17-4146-80cd-780ee050b283 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:37.038011 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60600bb-79d8-49f0-a4ac-2a5aad831227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:37.038608 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60600bb-79d8-49f0-a4ac-2a5aad831227 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:37.038883 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60600bb-79d8-49f0-a4ac-2a5aad831227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:37.039146 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60600bb-79d8-49f0-a4ac-2a5aad831227 admin admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:37.039822 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60600bb-79d8-49f0-a4ac-2a5aad831227 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:19:37.040123 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60600bb-79d8-49f0-a4ac-2a5aad831227 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:37.041730 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60600bb-79d8-49f0-a4ac-2a5aad831227 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:37.053237 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1414/2815] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:19:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:19:37.065798 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa19e7f-b12f-4f4b-8b97-454337bf5372 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:37.075903 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa19e7f-b12f-4f4b-8b97-454337bf5372 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:37.081722 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8228a31-a32d-4e32-be8e-8e66e29b7534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:37.082328 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8228a31-a32d-4e32-be8e-8e66e29b7534 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:37.082608 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8228a31-a32d-4e32-be8e-8e66e29b7534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:37.082877 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8228a31-a32d-4e32-be8e-8e66e29b7534 admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:37.083553 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8228a31-a32d-4e32-be8e-8e66e29b7534 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:37.085495 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8228a31-a32d-4e32-be8e-8e66e29b7534 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:37.090016 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:19:37.117315 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1402/2816] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:19:37 2022] POST /identity/v3/projects => generated 407 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:19:37.121881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60600bb-79d8-49f0-a4ac-2a5aad831227 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:37.133754 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60600bb-79d8-49f0-a4ac-2a5aad831227 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:37.157022 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:19:37.199836 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b0af2-be3c-44c8-b6bc-0e4feaea29ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:37.200667 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b0af2-be3c-44c8-b6bc-0e4feaea29ed admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:37.200667 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b0af2-be3c-44c8-b6bc-0e4feaea29ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:37.200667 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b0af2-be3c-44c8-b6bc-0e4feaea29ed admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:37.202715 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1b0af2-be3c-44c8-b6bc-0e4feaea29ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:37.204691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1b0af2-be3c-44c8-b6bc-0e4feaea29ed admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:37.228386 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1415/2817] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:19:37 2022] POST /identity/v3/users => generated 304 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:19:37.232835 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8228a31-a32d-4e32-be8e-8e66e29b7534 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:37.244655 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8228a31-a32d-4e32-be8e-8e66e29b7534 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:37.250726 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e30b9-c9db-4102-a4cf-dae1b064c5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:37.251400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e30b9-c9db-4102-a4cf-dae1b064c5fa admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:37.251805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e30b9-c9db-4102-a4cf-dae1b064c5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:37.252129 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e30b9-c9db-4102-a4cf-dae1b064c5fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:37.252956 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1e30b9-c9db-4102-a4cf-dae1b064c5fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:37.254617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1e30b9-c9db-4102-a4cf-dae1b064c5fa admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:37.266650 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1403/2818] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:19:37.271253 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b0af2-be3c-44c8-b6bc-0e4feaea29ed admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:37.287879 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b0af2-be3c-44c8-b6bc-0e4feaea29ed admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:37.293880 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64765f17-3193-4e5f-b9ae-27d38ce88ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:37.294490 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64765f17-3193-4e5f-b9ae-27d38ce88ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:37.294768 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64765f17-3193-4e5f-b9ae-27d38ce88ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:37.295069 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64765f17-3193-4e5f-b9ae-27d38ce88ee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:37.295759 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64765f17-3193-4e5f-b9ae-27d38ce88ee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:37.297404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64765f17-3193-4e5f-b9ae-27d38ce88ee3 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:37.308525 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1416/2819] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:19:37.316317 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e30b9-c9db-4102-a4cf-dae1b064c5fa admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:37.327326 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e30b9-c9db-4102-a4cf-dae1b064c5fa admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:37.333408 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed8b019-6254-4a02-84dc-40277c3147cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:37.334182 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8b019-6254-4a02-84dc-40277c3147cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:37.334540 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8b019-6254-4a02-84dc-40277c3147cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:37.334875 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8b019-6254-4a02-84dc-40277c3147cb admin admin] PATH_INFO: `/v3/projects/4cba44fd0ba4471d9de95661ee7af13f/users/c2fcdb80b6c44c168274107a26af6fa5/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:37.350658 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed8b019-6254-4a02-84dc-40277c3147cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cba44fd0ba4471d9de95661ee7af13f, user_id=c2fcdb80b6c44c168274107a26af6fa5, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:37.353331 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed8b019-6254-4a02-84dc-40277c3147cb admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:37.367969 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1404/2820] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:19:37 2022] PUT /identity/v3/projects/4cba44fd0ba4471d9de95661ee7af13f/users/c2fcdb80b6c44c168274107a26af6fa5/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:19:37.380396 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb4647-5007-4a7b-8229-5238ea705efa None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:37.380829 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb4647-5007-4a7b-8229-5238ea705efa None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:37.381171 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb4647-5007-4a7b-8229-5238ea705efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:37.417197 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-10cb4647-5007-4a7b-8229-5238ea705efa None None] MFA Rules not processed for user `c2fcdb80b6c44c168274107a26af6fa5`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:19:37.418937 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-10cb4647-5007-4a7b-8229-5238ea705efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:37.493336 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1417/2821] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:19:37 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:19:37.498901 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed8b019-6254-4a02-84dc-40277c3147cb admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:37.577993 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed8b019-6254-4a02-84dc-40277c3147cb admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:37.583544 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42572e-6ebd-4bcb-9354-8ca8e70dfabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:37.584328 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42572e-6ebd-4bcb-9354-8ca8e70dfabc admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:37.584694 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42572e-6ebd-4bcb-9354-8ca8e70dfabc admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:37.585087 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42572e-6ebd-4bcb-9354-8ca8e70dfabc admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:37.586051 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b42572e-6ebd-4bcb-9354-8ca8e70dfabc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:37.588085 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b42572e-6ebd-4bcb-9354-8ca8e70dfabc admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:37.622669 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1405/2822] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:19:37 2022] POST /identity/v3/projects => generated 409 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:19:37.628163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb4647-5007-4a7b-8229-5238ea705efa None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:37.640157 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb4647-5007-4a7b-8229-5238ea705efa None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:37.701762 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3234a-1968-48c0-9a39-ee967b68282a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:37.702375 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3234a-1968-48c0-9a39-ee967b68282a admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:37.702657 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3234a-1968-48c0-9a39-ee967b68282a admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:37.702922 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3234a-1968-48c0-9a39-ee967b68282a admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:37.703594 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece3234a-1968-48c0-9a39-ee967b68282a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:37.705536 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece3234a-1968-48c0-9a39-ee967b68282a admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:37.729503 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1418/2823] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:19:37 2022] POST /identity/v3/users => generated 311 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:19:37.735011 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42572e-6ebd-4bcb-9354-8ca8e70dfabc admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:37.745703 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42572e-6ebd-4bcb-9354-8ca8e70dfabc admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:37.753682 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb152f-442a-4a1d-a33b-4a9e9c4f6143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:37.754384 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb152f-442a-4a1d-a33b-4a9e9c4f6143 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:37.754707 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb152f-442a-4a1d-a33b-4a9e9c4f6143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:37.755042 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb152f-442a-4a1d-a33b-4a9e9c4f6143 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:37.755802 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddb152f-442a-4a1d-a33b-4a9e9c4f6143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:37.757611 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddb152f-442a-4a1d-a33b-4a9e9c4f6143 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:37.773900 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1406/2824] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:19:37.778318 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3234a-1968-48c0-9a39-ee967b68282a admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:37.789469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3234a-1968-48c0-9a39-ee967b68282a admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:37.797637 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f70e5-fef9-49d6-a02f-5cf867db4b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:37.798563 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f70e5-fef9-49d6-a02f-5cf867db4b8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:37.799612 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f70e5-fef9-49d6-a02f-5cf867db4b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:37.799612 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f70e5-fef9-49d6-a02f-5cf867db4b8d admin admin] PATH_INFO: `/v3/projects/f31673a4f49a49e49b9ec2d3eac263c5/users/51a0eae0923942e4af27229e6626d516/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:37.817075 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f70e5-fef9-49d6-a02f-5cf867db4b8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f31673a4f49a49e49b9ec2d3eac263c5, user_id=51a0eae0923942e4af27229e6626d516, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:37.820216 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f70e5-fef9-49d6-a02f-5cf867db4b8d admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:37.830730 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:19:37.839918 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1419/2825] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:19:37 2022] PUT /identity/v3/projects/f31673a4f49a49e49b9ec2d3eac263c5/users/51a0eae0923942e4af27229e6626d516/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:19:37.848370 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb152f-442a-4a1d-a33b-4a9e9c4f6143 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:37.860474 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb152f-442a-4a1d-a33b-4a9e9c4f6143 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:37.922018 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eaac0f-e5c7-44cd-8944-16c2f7aa0218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:37.922572 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eaac0f-e5c7-44cd-8944-16c2f7aa0218 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:37.922885 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eaac0f-e5c7-44cd-8944-16c2f7aa0218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:37.923255 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eaac0f-e5c7-44cd-8944-16c2f7aa0218 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:37.923998 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eaac0f-e5c7-44cd-8944-16c2f7aa0218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:37.925841 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eaac0f-e5c7-44cd-8944-16c2f7aa0218 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:37.936510 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1407/2826] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:19:37.943476 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f70e5-fef9-49d6-a02f-5cf867db4b8d admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:37.955491 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f70e5-fef9-49d6-a02f-5cf867db4b8d admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:37.960959 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2427d-173d-46eb-a20c-e3b77ec3eaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:37.961622 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2427d-173d-46eb-a20c-e3b77ec3eaaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:37.961896 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2427d-173d-46eb-a20c-e3b77ec3eaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:37.962169 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2427d-173d-46eb-a20c-e3b77ec3eaaa admin admin] PATH_INFO: `/v3/projects/f31673a4f49a49e49b9ec2d3eac263c5/users/51a0eae0923942e4af27229e6626d516/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:37.964158 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2427d-173d-46eb-a20c-e3b77ec3eaaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=f31673a4f49a49e49b9ec2d3eac263c5, user_id=51a0eae0923942e4af27229e6626d516, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:37.966925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2427d-173d-46eb-a20c-e3b77ec3eaaa admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:37.978318 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1420/2827] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:19:37 2022] PUT /identity/v3/projects/f31673a4f49a49e49b9ec2d3eac263c5/users/51a0eae0923942e4af27229e6626d516/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:19:38.045230 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6917dd43-7d83-4ae3-add9-a33c2d577a46 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:38.045890 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6917dd43-7d83-4ae3-add9-a33c2d577a46 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:38.045890 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6917dd43-7d83-4ae3-add9-a33c2d577a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:38.077202 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-6917dd43-7d83-4ae3-add9-a33c2d577a46 None None] MFA Rules not processed for user `51a0eae0923942e4af27229e6626d516`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:19:38.078411 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-6917dd43-7d83-4ae3-add9-a33c2d577a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:38.087320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:19:38.155515 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1408/2828] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:19:38 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:19:38.167163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2427d-173d-46eb-a20c-e3b77ec3eaaa admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:38.168925 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-f5e2427d-173d-46eb-a20c-e3b77ec3eaaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:38.236833 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2427d-173d-46eb-a20c-e3b77ec3eaaa admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:38.241250 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97430011-b02e-4a45-a601-f0e1ca38aa11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:38.241881 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97430011-b02e-4a45-a601-f0e1ca38aa11 service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:38.242348 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97430011-b02e-4a45-a601-f0e1ca38aa11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:38.242557 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97430011-b02e-4a45-a601-f0e1ca38aa11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:38.253600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97430011-b02e-4a45-a601-f0e1ca38aa11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:38.255138 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97430011-b02e-4a45-a601-f0e1ca38aa11 service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:38.265239 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1421/2829] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:19:38 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:19:38.464617 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6917dd43-7d83-4ae3-add9-a33c2d577a46 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:38.475574 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6917dd43-7d83-4ae3-add9-a33c2d577a46 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:38.479692 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939057f-1358-4855-9a13-f1f0c3e834bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:38.480412 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939057f-1358-4855-9a13-f1f0c3e834bc service cinder] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:38.480830 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939057f-1358-4855-9a13-f1f0c3e834bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:38.481211 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939057f-1358-4855-9a13-f1f0c3e834bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:38.550356 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1939057f-1358-4855-9a13-f1f0c3e834bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:38.551780 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1939057f-1358-4855-9a13-f1f0c3e834bc service cinder] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:38.561701 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1409/2830] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:19:38 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:19:47.249689 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97430011-b02e-4a45-a601-f0e1ca38aa11 service cinder] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:47.333343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97430011-b02e-4a45-a601-f0e1ca38aa11 service cinder] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:47.338786 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a6909-5e2f-45fe-aa63-c2cd698cf125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:47.339159 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a6909-5e2f-45fe-aa63-c2cd698cf125 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:47.339436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a6909-5e2f-45fe-aa63-c2cd698cf125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:47.339713 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a6909-5e2f-45fe-aa63-c2cd698cf125 admin admin] PATH_INFO: `/v3/users/c2fcdb80b6c44c168274107a26af6fa5` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:47.340954 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8a6909-5e2f-45fe-aa63-c2cd698cf125 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2fcdb80b6c44c168274107a26af6fa5)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:47.343350 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8a6909-5e2f-45fe-aa63-c2cd698cf125 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:47.405067 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-4c8a6909-5e2f-45fe-aa63-c2cd698cf125 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2fcdb80b6c44c168274107a26af6fa5'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:47.413771 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-4c8a6909-5e2f-45fe-aa63-c2cd698cf125 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2fcdb80b6c44c168274107a26af6fa5'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:47.422234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-4c8a6909-5e2f-45fe-aa63-c2cd698cf125 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2fcdb80b6c44c168274107a26af6fa5'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:47.435939 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-4c8a6909-5e2f-45fe-aa63-c2cd698cf125 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2fcdb80b6c44c168274107a26af6fa5'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:47.436757 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-4c8a6909-5e2f-45fe-aa63-c2cd698cf125 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2fcdb80b6c44c168274107a26af6fa5'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:47.443061 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1422/2831] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:19:47 2022] DELETE /identity/v3/users/c2fcdb80b6c44c168274107a26af6fa5 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:19:47.447775 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f02cc0-81da-4b8c-94c5-2500f2094073 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:47.448548 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f02cc0-81da-4b8c-94c5-2500f2094073 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:47.448548 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f02cc0-81da-4b8c-94c5-2500f2094073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:47.468525 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-c6f02cc0-81da-4b8c-94c5-2500f2094073 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:19:47.469780 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-c6f02cc0-81da-4b8c-94c5-2500f2094073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:47.541333 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1410/2832] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:19:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:19:47.552662 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a6909-5e2f-45fe-aa63-c2cd698cf125 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:47.554387 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-4c8a6909-5e2f-45fe-aa63-c2cd698cf125 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:47.567792 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:19:47.628837 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a6909-5e2f-45fe-aa63-c2cd698cf125 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:47.633337 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e64938-7874-410a-8712-59cce5bb981d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:47.633337 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e64938-7874-410a-8712-59cce5bb981d service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:47.633623 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e64938-7874-410a-8712-59cce5bb981d service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:47.633772 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e64938-7874-410a-8712-59cce5bb981d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:47.646990 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e64938-7874-410a-8712-59cce5bb981d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:47.648753 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e64938-7874-410a-8712-59cce5bb981d service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:47.660335 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1423/2833] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:19:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:19:47.983903 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f02cc0-81da-4b8c-94c5-2500f2094073 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:47.985457 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-c6f02cc0-81da-4b8c-94c5-2500f2094073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:48.003529 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f02cc0-81da-4b8c-94c5-2500f2094073 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:48.008537 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf9c23-c362-4133-810e-bfbefadef4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:48.009210 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf9c23-c362-4133-810e-bfbefadef4f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:48.009485 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf9c23-c362-4133-810e-bfbefadef4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:48.009755 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf9c23-c362-4133-810e-bfbefadef4f3 admin admin] PATH_INFO: `/v3/projects/4cba44fd0ba4471d9de95661ee7af13f` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:48.010936 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf9c23-c362-4133-810e-bfbefadef4f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cba44fd0ba4471d9de95661ee7af13f)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:48.012906 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf9c23-c362-4133-810e-bfbefadef4f3 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:48.061457 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:19:48.069969 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-bcdf9c23-c362-4133-810e-bfbefadef4f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cba44fd0ba4471d9de95661ee7af13f', 'request_id': 'req-bcdf9c23-c362-4133-810e-bfbefadef4f3'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:48.079013 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-bcdf9c23-c362-4133-810e-bfbefadef4f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cba44fd0ba4471d9de95661ee7af13f', 'request_id': 'req-bcdf9c23-c362-4133-810e-bfbefadef4f3'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:48.085852 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-bcdf9c23-c362-4133-810e-bfbefadef4f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cba44fd0ba4471d9de95661ee7af13f', 'request_id': 'req-bcdf9c23-c362-4133-810e-bfbefadef4f3'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:48.100418 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-bcdf9c23-c362-4133-810e-bfbefadef4f3 admin admin] The token cache is being invalidate because project 4cba44fd0ba4471d9de95661ee7af13f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:19:48.100769 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-bcdf9c23-c362-4133-810e-bfbefadef4f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:48.101469 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-bcdf9c23-c362-4133-810e-bfbefadef4f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:48.103061 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1411/2834] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:19:47 2022] DELETE /identity/v3/projects/4cba44fd0ba4471d9de95661ee7af13f => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:19:48.107195 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e64938-7874-410a-8712-59cce5bb981d service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:48.120154 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e64938-7874-410a-8712-59cce5bb981d service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:48.183366 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8660022d-ad0a-42a2-a8b6-947b74bdcee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:48.184326 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660022d-ad0a-42a2-a8b6-947b74bdcee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:48.184629 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660022d-ad0a-42a2-a8b6-947b74bdcee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:48.184928 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660022d-ad0a-42a2-a8b6-947b74bdcee1 admin admin] PATH_INFO: `/v3/users/51a0eae0923942e4af27229e6626d516` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:48.186574 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8660022d-ad0a-42a2-a8b6-947b74bdcee1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51a0eae0923942e4af27229e6626d516)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:48.188490 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8660022d-ad0a-42a2-a8b6-947b74bdcee1 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:48.249112 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-8660022d-ad0a-42a2-a8b6-947b74bdcee1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51a0eae0923942e4af27229e6626d516'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:48.257045 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-8660022d-ad0a-42a2-a8b6-947b74bdcee1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51a0eae0923942e4af27229e6626d516'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:48.266235 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-8660022d-ad0a-42a2-a8b6-947b74bdcee1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51a0eae0923942e4af27229e6626d516'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:48.295370 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-8660022d-ad0a-42a2-a8b6-947b74bdcee1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51a0eae0923942e4af27229e6626d516'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:48.296203 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-8660022d-ad0a-42a2-a8b6-947b74bdcee1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51a0eae0923942e4af27229e6626d516'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:48.303259 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1424/2835] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:19:48 2022] DELETE /identity/v3/users/51a0eae0923942e4af27229e6626d516 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:19:48.631946 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf9c23-c362-4133-810e-bfbefadef4f3 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:48.634095 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-bcdf9c23-c362-4133-810e-bfbefadef4f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:48.655066 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf9c23-c362-4133-810e-bfbefadef4f3 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:48.734260 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972db65e-8636-4bd8-aa58-d10a9d5d8cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:48.734805 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972db65e-8636-4bd8-aa58-d10a9d5d8cbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:48.735105 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972db65e-8636-4bd8-aa58-d10a9d5d8cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:48.735358 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972db65e-8636-4bd8-aa58-d10a9d5d8cbf admin admin] PATH_INFO: `/v3/projects/f31673a4f49a49e49b9ec2d3eac263c5` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:48.736516 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972db65e-8636-4bd8-aa58-d10a9d5d8cbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=f31673a4f49a49e49b9ec2d3eac263c5)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:48.738488 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972db65e-8636-4bd8-aa58-d10a9d5d8cbf admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:48.804186 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-972db65e-8636-4bd8-aa58-d10a9d5d8cbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f31673a4f49a49e49b9ec2d3eac263c5', 'request_id': 'req-972db65e-8636-4bd8-aa58-d10a9d5d8cbf'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:48.813372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-972db65e-8636-4bd8-aa58-d10a9d5d8cbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f31673a4f49a49e49b9ec2d3eac263c5', 'request_id': 'req-972db65e-8636-4bd8-aa58-d10a9d5d8cbf'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:48.820387 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-972db65e-8636-4bd8-aa58-d10a9d5d8cbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f31673a4f49a49e49b9ec2d3eac263c5', 'request_id': 'req-972db65e-8636-4bd8-aa58-d10a9d5d8cbf'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:48.832497 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-972db65e-8636-4bd8-aa58-d10a9d5d8cbf admin admin] The token cache is being invalidate because project f31673a4f49a49e49b9ec2d3eac263c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:19:48.832822 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-972db65e-8636-4bd8-aa58-d10a9d5d8cbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:48.833481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-972db65e-8636-4bd8-aa58-d10a9d5d8cbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:48.835010 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1412/2836] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:19:48 2022] DELETE /identity/v3/projects/f31673a4f49a49e49b9ec2d3eac263c5 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:19:48.839977 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e3f91-3fc1-47b6-a878-84799e1553ec None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:48.840281 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e3f91-3fc1-47b6-a878-84799e1553ec None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:48.840552 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e3f91-3fc1-47b6-a878-84799e1553ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:48.864954 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-db3e3f91-3fc1-47b6-a878-84799e1553ec None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:19:48.866169 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-db3e3f91-3fc1-47b6-a878-84799e1553ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:48.893366 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:19:48.941780 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1425/2837] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:19:48.949826 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094e09c-9fa8-4bd3-9622-83a5e4016f13 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:48.950105 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094e09c-9fa8-4bd3-9622-83a5e4016f13 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:48.950387 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094e09c-9fa8-4bd3-9622-83a5e4016f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:48.974103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-7094e09c-9fa8-4bd3-9622-83a5e4016f13 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:19:48.975491 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-7094e09c-9fa8-4bd3-9622-83a5e4016f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:48.984083 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1413/2838] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:19:48.988775 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e3f91-3fc1-47b6-a878-84799e1553ec None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:49.000289 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e3f91-3fc1-47b6-a878-84799e1553ec None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:49.004930 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18586bc-88ce-43ac-80db-84652a8f683b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:49.005572 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18586bc-88ce-43ac-80db-84652a8f683b admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:49.005852 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18586bc-88ce-43ac-80db-84652a8f683b admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:49.006122 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18586bc-88ce-43ac-80db-84652a8f683b admin admin] PATH_INFO: `/v3/domains` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:49.006818 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18586bc-88ce-43ac-80db-84652a8f683b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80478) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:19:49.007119 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18586bc-88ce-43ac-80db-84652a8f683b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:49.008735 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18586bc-88ce-43ac-80db-84652a8f683b admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:49.021025 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1426/2839] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:19:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 20:19:49.026599 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094e09c-9fa8-4bd3-9622-83a5e4016f13 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:49.042503 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094e09c-9fa8-4bd3-9622-83a5e4016f13 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:49.051185 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ae4b1-6481-4998-aace-12a28da2e577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:49.052169 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ae4b1-6481-4998-aace-12a28da2e577 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:49.052610 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ae4b1-6481-4998-aace-12a28da2e577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:49.053119 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ae4b1-6481-4998-aace-12a28da2e577 admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:49.054251 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ae4b1-6481-4998-aace-12a28da2e577 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:49.057744 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ae4b1-6481-4998-aace-12a28da2e577 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:49.099097 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1414/2840] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:19:49 2022] POST /identity/v3/projects => generated 381 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:19:49.103350 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18586bc-88ce-43ac-80db-84652a8f683b admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:49.114948 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18586bc-88ce-43ac-80db-84652a8f683b admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:49.181965 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce074e39-5549-4d50-ba5a-ec6a72538196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:49.182582 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce074e39-5549-4d50-ba5a-ec6a72538196 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:49.182857 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce074e39-5549-4d50-ba5a-ec6a72538196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:49.183128 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce074e39-5549-4d50-ba5a-ec6a72538196 admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:49.183796 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce074e39-5549-4d50-ba5a-ec6a72538196 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:49.190740 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce074e39-5549-4d50-ba5a-ec6a72538196 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:49.207821 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1427/2841] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:19:49 2022] POST /identity/v3/users => generated 291 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:19:49.212170 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ae4b1-6481-4998-aace-12a28da2e577 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:49.223888 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ae4b1-6481-4998-aace-12a28da2e577 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:49.231368 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a047d-6c1a-4e7b-9c0c-390457d98470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:49.231993 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a047d-6c1a-4e7b-9c0c-390457d98470 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:49.232884 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a047d-6c1a-4e7b-9c0c-390457d98470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:49.233293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a047d-6c1a-4e7b-9c0c-390457d98470 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:49.234145 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a047d-6c1a-4e7b-9c0c-390457d98470 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:49.235748 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a047d-6c1a-4e7b-9c0c-390457d98470 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:49.247315 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1415/2842] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:19:49.252036 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce074e39-5549-4d50-ba5a-ec6a72538196 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:49.262600 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce074e39-5549-4d50-ba5a-ec6a72538196 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:49.267344 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b01e4c-563a-4716-b149-71394ab252db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:49.267949 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b01e4c-563a-4716-b149-71394ab252db admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:49.268227 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b01e4c-563a-4716-b149-71394ab252db admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:49.268498 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b01e4c-563a-4716-b149-71394ab252db admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:49.269223 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b01e4c-563a-4716-b149-71394ab252db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:49.270813 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b01e4c-563a-4716-b149-71394ab252db admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:49.282530 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1428/2843] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:19:49.287021 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a047d-6c1a-4e7b-9c0c-390457d98470 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:49.299864 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a047d-6c1a-4e7b-9c0c-390457d98470 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:49.304536 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91a75a-2e40-465b-9155-9661736698f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:49.305244 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91a75a-2e40-465b-9155-9661736698f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:49.305521 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91a75a-2e40-465b-9155-9661736698f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:49.305793 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91a75a-2e40-465b-9155-9661736698f0 admin admin] PATH_INFO: `/v3/projects/ec6da54616744fc9a9f16f03ac791442/users/9b2cbd15c4894a628d251c90756d37c4/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:49.322658 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c91a75a-2e40-465b-9155-9661736698f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec6da54616744fc9a9f16f03ac791442, user_id=9b2cbd15c4894a628d251c90756d37c4, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:49.325332 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c91a75a-2e40-465b-9155-9661736698f0 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:49.327646 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:19:49.340564 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1416/2844] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:19:49 2022] PUT /identity/v3/projects/ec6da54616744fc9a9f16f03ac791442/users/9b2cbd15c4894a628d251c90756d37c4/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:19:49.351088 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744706d-8355-4ce9-a1e9-3abdb46826a5 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:49.351380 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744706d-8355-4ce9-a1e9-3abdb46826a5 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:49.351656 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744706d-8355-4ce9-a1e9-3abdb46826a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:49.383957 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-1744706d-8355-4ce9-a1e9-3abdb46826a5 None None] MFA Rules not processed for user `9b2cbd15c4894a628d251c90756d37c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:19:49.385232 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-1744706d-8355-4ce9-a1e9-3abdb46826a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:49.455544 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1429/2845] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:19:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:19:49.460490 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91a75a-2e40-465b-9155-9661736698f0 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:49.536847 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91a75a-2e40-465b-9155-9661736698f0 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:49.541590 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a66fc-2336-4d6c-bdca-126a46b4408e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:49.542207 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a66fc-2336-4d6c-bdca-126a46b4408e admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:49.542488 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a66fc-2336-4d6c-bdca-126a46b4408e admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:49.542760 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a66fc-2336-4d6c-bdca-126a46b4408e admin admin] PATH_INFO: `/v3/projects` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:49.543422 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a66fc-2336-4d6c-bdca-126a46b4408e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:49.545628 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a66fc-2336-4d6c-bdca-126a46b4408e admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:49.585249 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1417/2846] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:19:49 2022] POST /identity/v3/projects => generated 381 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:19:49.589436 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744706d-8355-4ce9-a1e9-3abdb46826a5 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:49.602044 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744706d-8355-4ce9-a1e9-3abdb46826a5 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:49.623108 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:19:49.669383 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6cb8a-b16a-4066-92cb-783df31248c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:49.670779 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6cb8a-b16a-4066-92cb-783df31248c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:49.670779 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6cb8a-b16a-4066-92cb-783df31248c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:49.670779 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6cb8a-b16a-4066-92cb-783df31248c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:49.671343 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb6cb8a-b16a-4066-92cb-783df31248c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:49.673369 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb6cb8a-b16a-4066-92cb-783df31248c8 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:49.699903 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1430/2847] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:19:49 2022] POST /identity/v3/users => generated 297 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:19:49.704633 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a66fc-2336-4d6c-bdca-126a46b4408e admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:49.721169 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a66fc-2336-4d6c-bdca-126a46b4408e admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:49.728790 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeae4b5-2f4a-4b00-9cda-33d9866b10b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:49.729803 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeae4b5-2f4a-4b00-9cda-33d9866b10b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:49.730300 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeae4b5-2f4a-4b00-9cda-33d9866b10b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:49.730813 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeae4b5-2f4a-4b00-9cda-33d9866b10b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:49.732060 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeae4b5-2f4a-4b00-9cda-33d9866b10b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:49.734623 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeae4b5-2f4a-4b00-9cda-33d9866b10b4 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:49.749935 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1418/2848] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:19:49.755024 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6cb8a-b16a-4066-92cb-783df31248c8 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:49.766131 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6cb8a-b16a-4066-92cb-783df31248c8 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:49.771065 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d22ccf-b001-44e3-9fba-ba69cb508c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:49.771765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d22ccf-b001-44e3-9fba-ba69cb508c0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:49.772043 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d22ccf-b001-44e3-9fba-ba69cb508c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:49.772602 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d22ccf-b001-44e3-9fba-ba69cb508c0b admin admin] PATH_INFO: `/v3/projects/3df5a4dd43c941fe8d0df5ed4a7c52f6/users/c6269b9adae545b985eaf24f312a6758/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:49.790484 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d22ccf-b001-44e3-9fba-ba69cb508c0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3df5a4dd43c941fe8d0df5ed4a7c52f6, user_id=c6269b9adae545b985eaf24f312a6758, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:49.793546 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d22ccf-b001-44e3-9fba-ba69cb508c0b admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:49.815089 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1431/2849] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:19:49 2022] PUT /identity/v3/projects/3df5a4dd43c941fe8d0df5ed4a7c52f6/users/c6269b9adae545b985eaf24f312a6758/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:19:49.820997 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeae4b5-2f4a-4b00-9cda-33d9866b10b4 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:49.832456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeae4b5-2f4a-4b00-9cda-33d9866b10b4 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:49.902493 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a24a83f-01e5-447f-bbb2-3190532ab0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:49.903141 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24a83f-01e5-447f-bbb2-3190532ab0c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:49.903417 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24a83f-01e5-447f-bbb2-3190532ab0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:49.903684 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24a83f-01e5-447f-bbb2-3190532ab0c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:49.905344 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a24a83f-01e5-447f-bbb2-3190532ab0c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:49.907002 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a24a83f-01e5-447f-bbb2-3190532ab0c3 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:49.917953 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1419/2850] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:19:49.923316 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d22ccf-b001-44e3-9fba-ba69cb508c0b admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:49.934142 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d22ccf-b001-44e3-9fba-ba69cb508c0b admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:49.940035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bccd6e8-6176-41ba-b8e8-8f60bbc4f8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:49.940749 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bccd6e8-6176-41ba-b8e8-8f60bbc4f8e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:49.941025 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bccd6e8-6176-41ba-b8e8-8f60bbc4f8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:49.941302 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bccd6e8-6176-41ba-b8e8-8f60bbc4f8e3 admin admin] PATH_INFO: `/v3/projects/3df5a4dd43c941fe8d0df5ed4a7c52f6/users/c6269b9adae545b985eaf24f312a6758/roles/89919f784ad144529b07c8ab92ee7d48` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:49.943310 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bccd6e8-6176-41ba-b8e8-8f60bbc4f8e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3df5a4dd43c941fe8d0df5ed4a7c52f6, user_id=c6269b9adae545b985eaf24f312a6758, role_id=89919f784ad144529b07c8ab92ee7d48)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:49.945992 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bccd6e8-6176-41ba-b8e8-8f60bbc4f8e3 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:49.959691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1432/2851] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:19:49 2022] PUT /identity/v3/projects/3df5a4dd43c941fe8d0df5ed4a7c52f6/users/c6269b9adae545b985eaf24f312a6758/roles/89919f784ad144529b07c8ab92ee7d48 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:19:49.976122 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507088b-1a9e-4c1a-b2aa-3bb4e0b0f2da None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:49.976456 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507088b-1a9e-4c1a-b2aa-3bb4e0b0f2da None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:49.976808 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507088b-1a9e-4c1a-b2aa-3bb4e0b0f2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:50.009842 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-9507088b-1a9e-4c1a-b2aa-3bb4e0b0f2da None None] MFA Rules not processed for user `c6269b9adae545b985eaf24f312a6758`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:19:50.011106 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-9507088b-1a9e-4c1a-b2aa-3bb4e0b0f2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:50.069890 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:19:50.093527 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1420/2852] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:19:49 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:19:50.105670 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bccd6e8-6176-41ba-b8e8-8f60bbc4f8e3 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:50.107849 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-3bccd6e8-6176-41ba-b8e8-8f60bbc4f8e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:50.197265 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bccd6e8-6176-41ba-b8e8-8f60bbc4f8e3 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:50.202929 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10865ae1-2a29-47d5-916a-779534f3bd7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:50.203927 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10865ae1-2a29-47d5-916a-779534f3bd7e service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:50.204405 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10865ae1-2a29-47d5-916a-779534f3bd7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:50.204889 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10865ae1-2a29-47d5-916a-779534f3bd7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:50.214969 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10865ae1-2a29-47d5-916a-779534f3bd7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:50.216636 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10865ae1-2a29-47d5-916a-779534f3bd7e service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:50.226736 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1433/2853] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:19:50 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:19:50.671967 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9507088b-1a9e-4c1a-b2aa-3bb4e0b0f2da None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:50.751507 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9507088b-1a9e-4c1a-b2aa-3bb4e0b0f2da None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:50.756602 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ec5240-c267-4ce1-b4c1-2516b0eb2523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:50.757364 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec5240-c267-4ce1-b4c1-2516b0eb2523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:50.757645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec5240-c267-4ce1-b4c1-2516b0eb2523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:50.757915 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec5240-c267-4ce1-b4c1-2516b0eb2523 admin admin] PATH_INFO: `/v3/users/9b2cbd15c4894a628d251c90756d37c4` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:50.759086 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ec5240-c267-4ce1-b4c1-2516b0eb2523 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b2cbd15c4894a628d251c90756d37c4)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:50.761032 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ec5240-c267-4ce1-b4c1-2516b0eb2523 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:50.822220 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-b6ec5240-c267-4ce1-b4c1-2516b0eb2523 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b2cbd15c4894a628d251c90756d37c4'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:50.831065 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-b6ec5240-c267-4ce1-b4c1-2516b0eb2523 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b2cbd15c4894a628d251c90756d37c4'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:50.849433 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-b6ec5240-c267-4ce1-b4c1-2516b0eb2523 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b2cbd15c4894a628d251c90756d37c4'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:50.850250 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-b6ec5240-c267-4ce1-b4c1-2516b0eb2523 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b2cbd15c4894a628d251c90756d37c4'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:50.859382 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-b6ec5240-c267-4ce1-b4c1-2516b0eb2523 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b2cbd15c4894a628d251c90756d37c4'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:50.867216 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1421/2854] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:19:50 2022] DELETE /identity/v3/users/9b2cbd15c4894a628d251c90756d37c4 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:19:50.874381 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345882b-9a43-4156-bdd0-e18b7ec6e744 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:50.875266 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345882b-9a43-4156-bdd0-e18b7ec6e744 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:50.875444 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345882b-9a43-4156-bdd0-e18b7ec6e744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:50.900306 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-2345882b-9a43-4156-bdd0-e18b7ec6e744 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:19:50.902437 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-2345882b-9a43-4156-bdd0-e18b7ec6e744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:50.981763 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1434/2855] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:19:50.993586 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ec5240-c267-4ce1-b4c1-2516b0eb2523 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:50.995177 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-b6ec5240-c267-4ce1-b4c1-2516b0eb2523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:51.078867 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ec5240-c267-4ce1-b4c1-2516b0eb2523 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:51.084302 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea233cc-7e2e-4aab-8ea6-5ed58aed2d85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:51.085197 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea233cc-7e2e-4aab-8ea6-5ed58aed2d85 service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:51.085667 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea233cc-7e2e-4aab-8ea6-5ed58aed2d85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:51.086103 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea233cc-7e2e-4aab-8ea6-5ed58aed2d85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:51.102360 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea233cc-7e2e-4aab-8ea6-5ed58aed2d85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:51.104954 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea233cc-7e2e-4aab-8ea6-5ed58aed2d85 service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:51.120000 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1422/2856] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:19:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:19:51.405535 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2345882b-9a43-4156-bdd0-e18b7ec6e744 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:51.407035 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-2345882b-9a43-4156-bdd0-e18b7ec6e744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:51.423109 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2345882b-9a43-4156-bdd0-e18b7ec6e744 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:51.428294 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5220875-0c27-4780-9624-9f37dc7b2d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:51.428977 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5220875-0c27-4780-9624-9f37dc7b2d47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:51.429253 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5220875-0c27-4780-9624-9f37dc7b2d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:51.429519 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5220875-0c27-4780-9624-9f37dc7b2d47 admin admin] PATH_INFO: `/v3/projects/ec6da54616744fc9a9f16f03ac791442` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:51.430679 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5220875-0c27-4780-9624-9f37dc7b2d47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec6da54616744fc9a9f16f03ac791442)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:51.432563 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5220875-0c27-4780-9624-9f37dc7b2d47 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:51.451273 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:19:51.484040 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-c5220875-0c27-4780-9624-9f37dc7b2d47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec6da54616744fc9a9f16f03ac791442', 'request_id': 'req-c5220875-0c27-4780-9624-9f37dc7b2d47'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:51.490302 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-c5220875-0c27-4780-9624-9f37dc7b2d47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec6da54616744fc9a9f16f03ac791442', 'request_id': 'req-c5220875-0c27-4780-9624-9f37dc7b2d47'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:51.498366 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-c5220875-0c27-4780-9624-9f37dc7b2d47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec6da54616744fc9a9f16f03ac791442', 'request_id': 'req-c5220875-0c27-4780-9624-9f37dc7b2d47'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:51.508906 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-c5220875-0c27-4780-9624-9f37dc7b2d47 admin admin] The token cache is being invalidate because project ec6da54616744fc9a9f16f03ac791442 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:19:51.509193 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-c5220875-0c27-4780-9624-9f37dc7b2d47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:51.509821 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-c5220875-0c27-4780-9624-9f37dc7b2d47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:51.511323 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1435/2857] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:19:51 2022] DELETE /identity/v3/projects/ec6da54616744fc9a9f16f03ac791442 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:19:51.516261 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea233cc-7e2e-4aab-8ea6-5ed58aed2d85 service neutron] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:51.536751 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea233cc-7e2e-4aab-8ea6-5ed58aed2d85 service neutron] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:51.547601 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:19:51.616078 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed41864d-d93b-48dc-854a-cfdee85b7153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:51.616427 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41864d-d93b-48dc-854a-cfdee85b7153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:51.616571 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41864d-d93b-48dc-854a-cfdee85b7153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:51.616891 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41864d-d93b-48dc-854a-cfdee85b7153 admin admin] PATH_INFO: `/v3/users/c6269b9adae545b985eaf24f312a6758` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:51.618020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed41864d-d93b-48dc-854a-cfdee85b7153 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6269b9adae545b985eaf24f312a6758)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:51.620024 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed41864d-d93b-48dc-854a-cfdee85b7153 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:51.692513 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ed41864d-d93b-48dc-854a-cfdee85b7153 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6269b9adae545b985eaf24f312a6758'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:51.701621 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ed41864d-d93b-48dc-854a-cfdee85b7153 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6269b9adae545b985eaf24f312a6758'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:51.719803 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ed41864d-d93b-48dc-854a-cfdee85b7153 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6269b9adae545b985eaf24f312a6758'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:51.720376 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ed41864d-d93b-48dc-854a-cfdee85b7153 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6269b9adae545b985eaf24f312a6758'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:51.727650 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-ed41864d-d93b-48dc-854a-cfdee85b7153 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6269b9adae545b985eaf24f312a6758'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:51.734393 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1423/2858] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:19:51 2022] DELETE /identity/v3/users/c6269b9adae545b985eaf24f312a6758 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:19:52.053982 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5220875-0c27-4780-9624-9f37dc7b2d47 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:52.055524 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-c5220875-0c27-4780-9624-9f37dc7b2d47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:52.070180 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5220875-0c27-4780-9624-9f37dc7b2d47 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:52.143616 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78625fc-e9ad-4bc2-bbcc-3ba6e9d0b65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:52.145417 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78625fc-e9ad-4bc2-bbcc-3ba6e9d0b65f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:52.145710 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78625fc-e9ad-4bc2-bbcc-3ba6e9d0b65f admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:52.146000 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78625fc-e9ad-4bc2-bbcc-3ba6e9d0b65f admin admin] PATH_INFO: `/v3/projects/3df5a4dd43c941fe8d0df5ed4a7c52f6` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:52.147573 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78625fc-e9ad-4bc2-bbcc-3ba6e9d0b65f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3df5a4dd43c941fe8d0df5ed4a7c52f6)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:52.149551 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78625fc-e9ad-4bc2-bbcc-3ba6e9d0b65f admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:52.202692 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-f78625fc-e9ad-4bc2-bbcc-3ba6e9d0b65f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3df5a4dd43c941fe8d0df5ed4a7c52f6', 'request_id': 'req-f78625fc-e9ad-4bc2-bbcc-3ba6e9d0b65f'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:52.209916 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-f78625fc-e9ad-4bc2-bbcc-3ba6e9d0b65f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3df5a4dd43c941fe8d0df5ed4a7c52f6', 'request_id': 'req-f78625fc-e9ad-4bc2-bbcc-3ba6e9d0b65f'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:52.220254 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-f78625fc-e9ad-4bc2-bbcc-3ba6e9d0b65f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3df5a4dd43c941fe8d0df5ed4a7c52f6', 'request_id': 'req-f78625fc-e9ad-4bc2-bbcc-3ba6e9d0b65f'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:52.236372 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-f78625fc-e9ad-4bc2-bbcc-3ba6e9d0b65f admin admin] The token cache is being invalidate because project 3df5a4dd43c941fe8d0df5ed4a7c52f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:19:52.236701 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-f78625fc-e9ad-4bc2-bbcc-3ba6e9d0b65f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:52.237404 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-f78625fc-e9ad-4bc2-bbcc-3ba6e9d0b65f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:52.238943 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1436/2859] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:19:52 2022] DELETE /identity/v3/projects/3df5a4dd43c941fe8d0df5ed4a7c52f6 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:19:52.244044 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b7d6d-3739-44ff-b396-2a20c04cce47 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:52.244862 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b7d6d-3739-44ff-b396-2a20c04cce47 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:52.244862 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b7d6d-3739-44ff-b396-2a20c04cce47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:52.265671 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-027b7d6d-3739-44ff-b396-2a20c04cce47 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:19:52.266985 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-027b7d6d-3739-44ff-b396-2a20c04cce47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:52.337379 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1424/2860] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:19:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:19:52.347570 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44266b-0642-4916-b97c-8315a70940e0 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:52.348007 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44266b-0642-4916-b97c-8315a70940e0 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:52.348334 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44266b-0642-4916-b97c-8315a70940e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:52.370236 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-ae44266b-0642-4916-b97c-8315a70940e0 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:19:52.371470 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-ae44266b-0642-4916-b97c-8315a70940e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:52.381133 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1437/2861] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:19:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:19:52.385563 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b7d6d-3739-44ff-b396-2a20c04cce47 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:52.397099 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b7d6d-3739-44ff-b396-2a20c04cce47 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:52.402560 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f86e7-db64-4409-9367-abc4a71659d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:52.403181 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f86e7-db64-4409-9367-abc4a71659d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:52.403460 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f86e7-db64-4409-9367-abc4a71659d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:52.404278 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f86e7-db64-4409-9367-abc4a71659d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:52.405020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f86e7-db64-4409-9367-abc4a71659d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:19:52.405328 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f86e7-db64-4409-9367-abc4a71659d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:52.406940 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f86e7-db64-4409-9367-abc4a71659d2 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:52.420293 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1425/2862] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:19:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:19:52.424941 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44266b-0642-4916-b97c-8315a70940e0 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:52.434738 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44266b-0642-4916-b97c-8315a70940e0 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:52.440097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852290be-81ca-4e54-b6ec-6ed173ca281f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:52.440756 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852290be-81ca-4e54-b6ec-6ed173ca281f admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:52.441036 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852290be-81ca-4e54-b6ec-6ed173ca281f admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:52.441470 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852290be-81ca-4e54-b6ec-6ed173ca281f admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:52.442208 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852290be-81ca-4e54-b6ec-6ed173ca281f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:52.444183 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852290be-81ca-4e54-b6ec-6ed173ca281f admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:52.479347 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1438/2863] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:19:52 2022] POST /identity/v3/projects => generated 361 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:19:52.483883 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f86e7-db64-4409-9367-abc4a71659d2 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:52.495278 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f86e7-db64-4409-9367-abc4a71659d2 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:52.510136 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:19:52.563316 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646f80c-0974-40a7-bdc8-ec27814d6d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:52.563934 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646f80c-0974-40a7-bdc8-ec27814d6d7b admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:52.564221 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646f80c-0974-40a7-bdc8-ec27814d6d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:52.564495 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646f80c-0974-40a7-bdc8-ec27814d6d7b admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:52.565274 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9646f80c-0974-40a7-bdc8-ec27814d6d7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:52.567140 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9646f80c-0974-40a7-bdc8-ec27814d6d7b admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:52.591206 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1426/2864] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:19:52 2022] POST /identity/v3/users => generated 281 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 20:19:52.596016 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852290be-81ca-4e54-b6ec-6ed173ca281f admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:52.603264 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:19:52.608059 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852290be-81ca-4e54-b6ec-6ed173ca281f admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:52.626144 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e2cec-c052-4bb6-9b47-670459134069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:52.626787 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e2cec-c052-4bb6-9b47-670459134069 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:52.627072 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e2cec-c052-4bb6-9b47-670459134069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:52.627344 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e2cec-c052-4bb6-9b47-670459134069 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:52.628053 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e2cec-c052-4bb6-9b47-670459134069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:52.629787 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e2cec-c052-4bb6-9b47-670459134069 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:52.644075 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1439/2865] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:19:52.649168 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646f80c-0974-40a7-bdc8-ec27814d6d7b admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:52.661498 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646f80c-0974-40a7-bdc8-ec27814d6d7b admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:52.666678 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968d535-85b8-4155-a92b-f8bf833d9d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:52.667315 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968d535-85b8-4155-a92b-f8bf833d9d9e admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:52.667593 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968d535-85b8-4155-a92b-f8bf833d9d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:52.667865 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968d535-85b8-4155-a92b-f8bf833d9d9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:52.668566 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0968d535-85b8-4155-a92b-f8bf833d9d9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:52.670252 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0968d535-85b8-4155-a92b-f8bf833d9d9e admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:52.682689 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1427/2866] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:19:52.687497 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e2cec-c052-4bb6-9b47-670459134069 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:52.699067 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e2cec-c052-4bb6-9b47-670459134069 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:52.707469 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6ba7f-0ea0-4c06-94bf-c275625bd01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:52.708595 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6ba7f-0ea0-4c06-94bf-c275625bd01a admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:52.709126 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6ba7f-0ea0-4c06-94bf-c275625bd01a admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:52.709597 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6ba7f-0ea0-4c06-94bf-c275625bd01a admin admin] PATH_INFO: `/v3/projects/77eb35c94bdd49b3b1786eed18900ecb/users/6faf8c19b01b4f91aad777907325c580/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:52.738066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a6ba7f-0ea0-4c06-94bf-c275625bd01a admin admin] RBAC: Authorizing `identity:create_grant(project_id=77eb35c94bdd49b3b1786eed18900ecb, user_id=6faf8c19b01b4f91aad777907325c580, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:52.743042 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a6ba7f-0ea0-4c06-94bf-c275625bd01a admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:52.767309 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1440/2867] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:19:52 2022] PUT /identity/v3/projects/77eb35c94bdd49b3b1786eed18900ecb/users/6faf8c19b01b4f91aad777907325c580/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:19:52.782554 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a4330-82d9-44a4-9616-bba2eb2f567a None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:52.783047 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a4330-82d9-44a4-9616-bba2eb2f567a None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:52.783517 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a4330-82d9-44a4-9616-bba2eb2f567a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:52.822104 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-e02a4330-82d9-44a4-9616-bba2eb2f567a None None] MFA Rules not processed for user `6faf8c19b01b4f91aad777907325c580`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:19:52.823374 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-e02a4330-82d9-44a4-9616-bba2eb2f567a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:52.904371 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1428/2868] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:19:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:19:52.942301 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6ba7f-0ea0-4c06-94bf-c275625bd01a admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:52.944340 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e2a6ba7f-0ea0-4c06-94bf-c275625bd01a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:53.006958 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6ba7f-0ea0-4c06-94bf-c275625bd01a admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:53.010489 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e06bf-7be3-4d82-91a5-4a28793bedf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:53.011106 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e06bf-7be3-4d82-91a5-4a28793bedf9 service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:53.011830 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e06bf-7be3-4d82-91a5-4a28793bedf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:53.011830 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e06bf-7be3-4d82-91a5-4a28793bedf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:53.021457 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192e06bf-7be3-4d82-91a5-4a28793bedf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:53.023259 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192e06bf-7be3-4d82-91a5-4a28793bedf9 service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:53.034629 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1441/2869] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:19:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:19:53.051035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a4330-82d9-44a4-9616-bba2eb2f567a None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:53.136512 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a4330-82d9-44a4-9616-bba2eb2f567a None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:53.143090 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a5533-3d4f-4934-a465-a5380eba7d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:53.143938 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a5533-3d4f-4934-a465-a5380eba7d9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:53.144225 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a5533-3d4f-4934-a465-a5380eba7d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:53.144508 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a5533-3d4f-4934-a465-a5380eba7d9c admin admin] PATH_INFO: `/v3/users/6faf8c19b01b4f91aad777907325c580` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:53.145908 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96a5533-3d4f-4934-a465-a5380eba7d9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6faf8c19b01b4f91aad777907325c580)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:53.148928 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96a5533-3d4f-4934-a465-a5380eba7d9c admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:53.181313 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:19:53.201585 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-f96a5533-3d4f-4934-a465-a5380eba7d9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6faf8c19b01b4f91aad777907325c580'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:53.210062 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-f96a5533-3d4f-4934-a465-a5380eba7d9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6faf8c19b01b4f91aad777907325c580'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:53.223516 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-f96a5533-3d4f-4934-a465-a5380eba7d9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6faf8c19b01b4f91aad777907325c580'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:53.224253 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-f96a5533-3d4f-4934-a465-a5380eba7d9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6faf8c19b01b4f91aad777907325c580'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:53.232518 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-f96a5533-3d4f-4934-a465-a5380eba7d9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6faf8c19b01b4f91aad777907325c580'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:53.238889 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1429/2870] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:19:53 2022] DELETE /identity/v3/users/6faf8c19b01b4f91aad777907325c580 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:19:53.243541 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f6655-fc0c-47b8-8fe8-78efe0b7da71 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:53.244266 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f6655-fc0c-47b8-8fe8-78efe0b7da71 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:53.244266 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f6655-fc0c-47b8-8fe8-78efe0b7da71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:53.264228 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-da8f6655-fc0c-47b8-8fe8-78efe0b7da71 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:19:53.265327 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-da8f6655-fc0c-47b8-8fe8-78efe0b7da71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:53.335372 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1442/2871] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:19:53.348876 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a5533-3d4f-4934-a465-a5380eba7d9c admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:53.350516 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-f96a5533-3d4f-4934-a465-a5380eba7d9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:53.422397 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a5533-3d4f-4934-a465-a5380eba7d9c admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:53.426587 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9461ad1-73e6-4571-8aaa-0e885b0c671f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:53.427223 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9461ad1-73e6-4571-8aaa-0e885b0c671f service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:53.427500 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9461ad1-73e6-4571-8aaa-0e885b0c671f service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:53.427772 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9461ad1-73e6-4571-8aaa-0e885b0c671f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:53.438328 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9461ad1-73e6-4571-8aaa-0e885b0c671f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:53.439810 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9461ad1-73e6-4571-8aaa-0e885b0c671f service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:53.451089 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1430/2872] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:19:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:19:53.793108 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f6655-fc0c-47b8-8fe8-78efe0b7da71 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:53.794799 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-da8f6655-fc0c-47b8-8fe8-78efe0b7da71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:53.813566 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f6655-fc0c-47b8-8fe8-78efe0b7da71 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:53.820187 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f1cea-7cb9-4fae-8e1a-bb2f7f8d95d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:53.821091 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f1cea-7cb9-4fae-8e1a-bb2f7f8d95d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:53.821609 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f1cea-7cb9-4fae-8e1a-bb2f7f8d95d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:53.821974 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f1cea-7cb9-4fae-8e1a-bb2f7f8d95d6 admin admin] PATH_INFO: `/v3/projects/77eb35c94bdd49b3b1786eed18900ecb` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:53.823783 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f1cea-7cb9-4fae-8e1a-bb2f7f8d95d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77eb35c94bdd49b3b1786eed18900ecb)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:53.826907 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f1cea-7cb9-4fae-8e1a-bb2f7f8d95d6 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:53.887221 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-cc1f1cea-7cb9-4fae-8e1a-bb2f7f8d95d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77eb35c94bdd49b3b1786eed18900ecb', 'request_id': 'req-cc1f1cea-7cb9-4fae-8e1a-bb2f7f8d95d6'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:53.895284 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-cc1f1cea-7cb9-4fae-8e1a-bb2f7f8d95d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77eb35c94bdd49b3b1786eed18900ecb', 'request_id': 'req-cc1f1cea-7cb9-4fae-8e1a-bb2f7f8d95d6'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:53.903285 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-cc1f1cea-7cb9-4fae-8e1a-bb2f7f8d95d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77eb35c94bdd49b3b1786eed18900ecb', 'request_id': 'req-cc1f1cea-7cb9-4fae-8e1a-bb2f7f8d95d6'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:53.909097 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:19:53.916562 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-cc1f1cea-7cb9-4fae-8e1a-bb2f7f8d95d6 admin admin] The token cache is being invalidate because project 77eb35c94bdd49b3b1786eed18900ecb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:19:53.916877 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-cc1f1cea-7cb9-4fae-8e1a-bb2f7f8d95d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:53.918167 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-cc1f1cea-7cb9-4fae-8e1a-bb2f7f8d95d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:19:53.922710 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1443/2873] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:19:53 2022] DELETE /identity/v3/projects/77eb35c94bdd49b3b1786eed18900ecb => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:19:53.929501 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0afc4-7664-4cfe-8773-fed164599aa9 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:53.929836 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0afc4-7664-4cfe-8773-fed164599aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:53.930289 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0afc4-7664-4cfe-8773-fed164599aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:53.952515 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-37d0afc4-7664-4cfe-8773-fed164599aa9 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:19:53.953790 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-37d0afc4-7664-4cfe-8773-fed164599aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:54.021431 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1431/2874] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:19:54.030045 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89dfb1-68ee-4883-9c77-e481795b4fcc None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:54.030337 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89dfb1-68ee-4883-9c77-e481795b4fcc None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:54.030618 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89dfb1-68ee-4883-9c77-e481795b4fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:54.053039 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-da89dfb1-68ee-4883-9c77-e481795b4fcc None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:19:54.054332 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-da89dfb1-68ee-4883-9c77-e481795b4fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:54.063338 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1444/2875] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:19:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:19:54.068013 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0afc4-7664-4cfe-8773-fed164599aa9 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:54.079577 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0afc4-7664-4cfe-8773-fed164599aa9 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:54.088235 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe16ce-a417-40cd-a457-a9ea70ad0653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:54.089096 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe16ce-a417-40cd-a457-a9ea70ad0653 admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:54.089514 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe16ce-a417-40cd-a457-a9ea70ad0653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:54.089801 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe16ce-a417-40cd-a457-a9ea70ad0653 admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:54.090536 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe16ce-a417-40cd-a457-a9ea70ad0653 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:19:54.090878 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe16ce-a417-40cd-a457-a9ea70ad0653 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:54.092758 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe16ce-a417-40cd-a457-a9ea70ad0653 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:54.106058 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1432/2876] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:19:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 20:19:54.114608 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89dfb1-68ee-4883-9c77-e481795b4fcc None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:54.126856 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89dfb1-68ee-4883-9c77-e481795b4fcc None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:54.131986 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afd8b1-21d5-4507-aec9-c344bb2911eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:54.132657 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afd8b1-21d5-4507-aec9-c344bb2911eb admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:54.132949 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afd8b1-21d5-4507-aec9-c344bb2911eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:54.133223 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afd8b1-21d5-4507-aec9-c344bb2911eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:54.133902 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3afd8b1-21d5-4507-aec9-c344bb2911eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:54.135841 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3afd8b1-21d5-4507-aec9-c344bb2911eb admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:54.168543 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1445/2877] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:19:54 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:19:54.174723 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe16ce-a417-40cd-a457-a9ea70ad0653 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:54.190465 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe16ce-a417-40cd-a457-a9ea70ad0653 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:54.241526 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:19:54.280900 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1a11e-97ae-407a-9281-bf21ca81f771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:54.281665 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1a11e-97ae-407a-9281-bf21ca81f771 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:54.282008 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1a11e-97ae-407a-9281-bf21ca81f771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:54.282445 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1a11e-97ae-407a-9281-bf21ca81f771 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:54.283332 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1a11e-97ae-407a-9281-bf21ca81f771 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:54.285703 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1a11e-97ae-407a-9281-bf21ca81f771 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:54.310872 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1433/2878] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:19:54 2022] POST /identity/v3/users => generated 285 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:19:54.315212 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afd8b1-21d5-4507-aec9-c344bb2911eb admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:54.326259 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afd8b1-21d5-4507-aec9-c344bb2911eb admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:54.335765 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f267fd-05c3-4649-8d19-bf845b80f4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:54.336359 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f267fd-05c3-4649-8d19-bf845b80f4f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:54.336702 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f267fd-05c3-4649-8d19-bf845b80f4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:54.336942 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f267fd-05c3-4649-8d19-bf845b80f4f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:54.337652 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f267fd-05c3-4649-8d19-bf845b80f4f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:54.340519 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f267fd-05c3-4649-8d19-bf845b80f4f8 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:54.352768 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1446/2879] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 20:19:54.357474 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1a11e-97ae-407a-9281-bf21ca81f771 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:54.368038 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1a11e-97ae-407a-9281-bf21ca81f771 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:54.373481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42222232-ec70-437a-937f-f3fa0dc7e74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:54.373481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42222232-ec70-437a-937f-f3fa0dc7e74e admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:54.373481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42222232-ec70-437a-937f-f3fa0dc7e74e admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:54.373481 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42222232-ec70-437a-937f-f3fa0dc7e74e admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:54.376632 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42222232-ec70-437a-937f-f3fa0dc7e74e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:54.376632 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42222232-ec70-437a-937f-f3fa0dc7e74e admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:54.387974 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1434/2880] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:19:54.394517 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f267fd-05c3-4649-8d19-bf845b80f4f8 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:54.406699 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f267fd-05c3-4649-8d19-bf845b80f4f8 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:54.412653 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788e448-8a9f-4479-9c03-93d2e3e14e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:54.413410 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788e448-8a9f-4479-9c03-93d2e3e14e46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:54.413691 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788e448-8a9f-4479-9c03-93d2e3e14e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:54.413975 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788e448-8a9f-4479-9c03-93d2e3e14e46 admin admin] PATH_INFO: `/v3/projects/db3eab4b37ca401b99cb2759893008d0/users/6644e243714f42e4b538be50e1eda336/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:54.432698 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788e448-8a9f-4479-9c03-93d2e3e14e46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db3eab4b37ca401b99cb2759893008d0, user_id=6644e243714f42e4b538be50e1eda336, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:54.435296 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788e448-8a9f-4479-9c03-93d2e3e14e46 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:54.454775 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1447/2881] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:19:54 2022] PUT /identity/v3/projects/db3eab4b37ca401b99cb2759893008d0/users/6644e243714f42e4b538be50e1eda336/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:19:54.473108 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9c1df-dd5a-49b8-9128-6fcb53623f97 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:54.473445 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9c1df-dd5a-49b8-9128-6fcb53623f97 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:54.473641 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9c1df-dd5a-49b8-9128-6fcb53623f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:54.505322 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-23e9c1df-dd5a-49b8-9128-6fcb53623f97 None None] MFA Rules not processed for user `6644e243714f42e4b538be50e1eda336`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:19:54.506568 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-23e9c1df-dd5a-49b8-9128-6fcb53623f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:54.588632 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1435/2882] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:19:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 20:19:54.605356 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788e448-8a9f-4479-9c03-93d2e3e14e46 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:19:54.607393 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-7788e448-8a9f-4479-9c03-93d2e3e14e46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:19:54.696308 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788e448-8a9f-4479-9c03-93d2e3e14e46 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:19:54.700170 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5374513-e6fa-4865-bf53-8aa136645e81 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe69b398c1f4e18b625a63ae40b01e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:19:54.701542 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5374513-e6fa-4865-bf53-8aa136645e81 service glance] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:19:54.702284 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5374513-e6fa-4865-bf53-8aa136645e81 service glance] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:19:54.702564 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5374513-e6fa-4865-bf53-8aa136645e81 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:19:54.714340 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5374513-e6fa-4865-bf53-8aa136645e81 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:19:54.715802 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5374513-e6fa-4865-bf53-8aa136645e81 service glance] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:19:54.729428 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1448/2883] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:19:54 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:20:04.584990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=50,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:20:04.724850 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=64,keystone:INSERT=2 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:20:27.003245 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e9c1df-dd5a-49b8-9128-6fcb53623f97 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:20:27.068182 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e9c1df-dd5a-49b8-9128-6fcb53623f97 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:20:27.073130 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4dec1-0c2d-4426-b15d-797f2eebe0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:20:27.073767 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4dec1-0c2d-4426-b15d-797f2eebe0bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:20:27.074040 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4dec1-0c2d-4426-b15d-797f2eebe0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:20:27.074310 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4dec1-0c2d-4426-b15d-797f2eebe0bc admin admin] PATH_INFO: `/v3/users/6644e243714f42e4b538be50e1eda336` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:20:27.075471 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4dec1-0c2d-4426-b15d-797f2eebe0bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6644e243714f42e4b538be50e1eda336)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:20:27.077645 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4dec1-0c2d-4426-b15d-797f2eebe0bc admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:20:27.128639 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-f8c4dec1-0c2d-4426-b15d-797f2eebe0bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6644e243714f42e4b538be50e1eda336'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:20:27.136550 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-f8c4dec1-0c2d-4426-b15d-797f2eebe0bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6644e243714f42e4b538be50e1eda336'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:20:27.149150 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-f8c4dec1-0c2d-4426-b15d-797f2eebe0bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6644e243714f42e4b538be50e1eda336'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:20:27.149829 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-f8c4dec1-0c2d-4426-b15d-797f2eebe0bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6644e243714f42e4b538be50e1eda336'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:20:27.161483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-f8c4dec1-0c2d-4426-b15d-797f2eebe0bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6644e243714f42e4b538be50e1eda336'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:20:27.167929 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1436/2884] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:20:27 2022] DELETE /identity/v3/users/6644e243714f42e4b538be50e1eda336 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:20:27.173737 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bf9e0-890b-4a55-b892-9d96aab8f562 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:20:27.174892 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bf9e0-890b-4a55-b892-9d96aab8f562 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:20:27.174892 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bf9e0-890b-4a55-b892-9d96aab8f562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:20:27.195837 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-e65bf9e0-890b-4a55-b892-9d96aab8f562 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:20:27.197068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e65bf9e0-890b-4a55-b892-9d96aab8f562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:20:27.275565 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1449/2885] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:20:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:20:27.286572 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4dec1-0c2d-4426-b15d-797f2eebe0bc admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:20:27.288001 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-f8c4dec1-0c2d-4426-b15d-797f2eebe0bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:20:27.355910 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4dec1-0c2d-4426-b15d-797f2eebe0bc admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:20:27.360161 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f73c2-ed25-495a-9a82-b903e8c0ab70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:20:27.360811 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f73c2-ed25-495a-9a82-b903e8c0ab70 service neutron] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:20:27.361094 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f73c2-ed25-495a-9a82-b903e8c0ab70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:20:27.361369 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f73c2-ed25-495a-9a82-b903e8c0ab70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:20:27.370799 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f73c2-ed25-495a-9a82-b903e8c0ab70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:20:27.372331 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f73c2-ed25-495a-9a82-b903e8c0ab70 service neutron] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:20:27.382134 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1437/2886] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:20:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:20:27.666259 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bf9e0-890b-4a55-b892-9d96aab8f562 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:20:27.669115 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e65bf9e0-890b-4a55-b892-9d96aab8f562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:20:27.685475 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bf9e0-890b-4a55-b892-9d96aab8f562 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:20:27.691887 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf157866-2ee6-43e9-ba42-15b345045cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:20:27.692531 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf157866-2ee6-43e9-ba42-15b345045cd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:20:27.692928 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf157866-2ee6-43e9-ba42-15b345045cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:20:27.693104 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf157866-2ee6-43e9-ba42-15b345045cd7 admin admin] PATH_INFO: `/v3/projects/db3eab4b37ca401b99cb2759893008d0` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:20:27.694361 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf157866-2ee6-43e9-ba42-15b345045cd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db3eab4b37ca401b99cb2759893008d0)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:20:27.696240 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf157866-2ee6-43e9-ba42-15b345045cd7 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:20:27.744254 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-bf157866-2ee6-43e9-ba42-15b345045cd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db3eab4b37ca401b99cb2759893008d0', 'request_id': 'req-bf157866-2ee6-43e9-ba42-15b345045cd7'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:20:27.750417 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-bf157866-2ee6-43e9-ba42-15b345045cd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db3eab4b37ca401b99cb2759893008d0', 'request_id': 'req-bf157866-2ee6-43e9-ba42-15b345045cd7'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:20:27.757955 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-bf157866-2ee6-43e9-ba42-15b345045cd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db3eab4b37ca401b99cb2759893008d0', 'request_id': 'req-bf157866-2ee6-43e9-ba42-15b345045cd7'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:20:27.770066 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-bf157866-2ee6-43e9-ba42-15b345045cd7 admin admin] The token cache is being invalidate because project db3eab4b37ca401b99cb2759893008d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80478) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:20:27.770354 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-bf157866-2ee6-43e9-ba42-15b345045cd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:20:27.770997 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-bf157866-2ee6-43e9-ba42-15b345045cd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:20:27.772804 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1450/2887] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:20:27 2022] DELETE /identity/v3/projects/db3eab4b37ca401b99cb2759893008d0 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:20:27.777627 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a04a8a-9f7f-4941-b1a5-47b71c98c265 None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:20:27.777910 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a04a8a-9f7f-4941-b1a5-47b71c98c265 None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:20:27.778182 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a04a8a-9f7f-4941-b1a5-47b71c98c265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:20:27.798840 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-a1a04a8a-9f7f-4941-b1a5-47b71c98c265 None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:20:27.800035 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-a1a04a8a-9f7f-4941-b1a5-47b71c98c265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:20:27.818050 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:20:27.861423 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1438/2888] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:20:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:20:27.869030 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23848ebf-398c-4fb7-8a21-789bb9f3e1eb None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:20:27.869314 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23848ebf-398c-4fb7-8a21-789bb9f3e1eb None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:20:27.869586 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23848ebf-398c-4fb7-8a21-789bb9f3e1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:20:27.889609 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-23848ebf-398c-4fb7-8a21-789bb9f3e1eb None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:20:27.890903 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-23848ebf-398c-4fb7-8a21-789bb9f3e1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:20:27.898929 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1451/2889] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:20:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:20:27.902990 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a04a8a-9f7f-4941-b1a5-47b71c98c265 None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:20:27.913238 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a04a8a-9f7f-4941-b1a5-47b71c98c265 None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:20:27.918200 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459212e-4166-4eb9-a42c-48809828aa7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:20:27.918797 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459212e-4166-4eb9-a42c-48809828aa7e admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:20:27.919073 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459212e-4166-4eb9-a42c-48809828aa7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:20:27.919343 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459212e-4166-4eb9-a42c-48809828aa7e admin admin] PATH_INFO: `/v3/domains` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:20:27.920021 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6459212e-4166-4eb9-a42c-48809828aa7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80477) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 20:20:27.920418 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6459212e-4166-4eb9-a42c-48809828aa7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:20:27.922146 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6459212e-4166-4eb9-a42c-48809828aa7e admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:20:27.933569 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1439/2890] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Aug 5 20:20:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 20:20:27.938567 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23848ebf-398c-4fb7-8a21-789bb9f3e1eb None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:20:27.947980 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23848ebf-398c-4fb7-8a21-789bb9f3e1eb None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:20:27.953501 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7251d6a-e676-4b20-a60e-ce05453f471a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:20:27.954101 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7251d6a-e676-4b20-a60e-ce05453f471a admin admin] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:20:27.954385 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7251d6a-e676-4b20-a60e-ce05453f471a admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:20:27.954659 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7251d6a-e676-4b20-a60e-ce05453f471a admin admin] PATH_INFO: `/v3/projects` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:20:27.955326 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7251d6a-e676-4b20-a60e-ce05453f471a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:20:27.957278 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7251d6a-e676-4b20-a60e-ce05453f471a admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:20:27.984493 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1452/2891] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Aug 5 20:20:27 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:20:27.988836 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459212e-4166-4eb9-a42c-48809828aa7e admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:20:27.999113 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459212e-4166-4eb9-a42c-48809828aa7e admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:20:28.058913 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1c43e-9a5c-46da-8fd6-c7c2fd643c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:20:28.059614 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1c43e-9a5c-46da-8fd6-c7c2fd643c76 admin admin] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:20:28.059842 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1c43e-9a5c-46da-8fd6-c7c2fd643c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:20:28.060116 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1c43e-9a5c-46da-8fd6-c7c2fd643c76 admin admin] PATH_INFO: `/v3/users` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:20:28.060812 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a1c43e-9a5c-46da-8fd6-c7c2fd643c76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:20:28.063613 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a1c43e-9a5c-46da-8fd6-c7c2fd643c76 admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:20:28.084444 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1440/2892] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Aug 5 20:20:27 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 20:20:28.088509 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7251d6a-e676-4b20-a60e-ce05453f471a admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:20:28.099087 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7251d6a-e676-4b20-a60e-ce05453f471a admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:20:28.103685 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b076554-a1c3-4072-b2de-ab81d83f84d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:20:28.104283 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b076554-a1c3-4072-b2de-ab81d83f84d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:20:28.104562 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b076554-a1c3-4072-b2de-ab81d83f84d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:20:28.104868 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b076554-a1c3-4072-b2de-ab81d83f84d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:20:28.105564 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b076554-a1c3-4072-b2de-ab81d83f84d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:20:28.107145 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b076554-a1c3-4072-b2de-ab81d83f84d9 admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:20:28.117848 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1453/2893] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:20:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:20:28.121747 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1c43e-9a5c-46da-8fd6-c7c2fd643c76 admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:20:28.132403 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1c43e-9a5c-46da-8fd6-c7c2fd643c76 admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:20:28.136974 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b1bab-91a5-4bcb-b312-a025b1ff146a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:20:28.137567 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b1bab-91a5-4bcb-b312-a025b1ff146a admin admin] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:20:28.137843 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b1bab-91a5-4bcb-b312-a025b1ff146a admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:20:28.138113 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b1bab-91a5-4bcb-b312-a025b1ff146a admin admin] PATH_INFO: `/v3/roles` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:20:28.138790 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b1bab-91a5-4bcb-b312-a025b1ff146a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:20:28.141074 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b1bab-91a5-4bcb-b312-a025b1ff146a admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:20:28.153020 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1441/2894] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Aug 5 20:20:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 20:20:28.157224 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b076554-a1c3-4072-b2de-ab81d83f84d9 admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:20:28.167169 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b076554-a1c3-4072-b2de-ab81d83f84d9 admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:20:28.172304 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44890082-fc23-41d2-b866-2cb47ca721cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:20:28.172983 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44890082-fc23-41d2-b866-2cb47ca721cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:20:28.173261 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44890082-fc23-41d2-b866-2cb47ca721cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:20:28.173535 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44890082-fc23-41d2-b866-2cb47ca721cf admin admin] PATH_INFO: `/v3/projects/e6c662e324bb4327af67b163bb03779e/users/7bf88e68a7fa4c898e707126aecc8837/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:20:28.188854 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44890082-fc23-41d2-b866-2cb47ca721cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6c662e324bb4327af67b163bb03779e, user_id=7bf88e68a7fa4c898e707126aecc8837, role_id=fbd7c81b4fba4b6d9e61b1e9bb3e2046)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:20:28.191495 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44890082-fc23-41d2-b866-2cb47ca721cf admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:20:28.205326 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1454/2895] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Aug 5 20:20:28 2022] PUT /identity/v3/projects/e6c662e324bb4327af67b163bb03779e/users/7bf88e68a7fa4c898e707126aecc8837/roles/fbd7c81b4fba4b6d9e61b1e9bb3e2046 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:20:28.215506 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc7aa5-1003-4d68-8bef-99c6191b966b None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:20:28.215787 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc7aa5-1003-4d68-8bef-99c6191b966b None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:20:28.216055 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc7aa5-1003-4d68-8bef-99c6191b966b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:20:28.243193 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-dffc7aa5-1003-4d68-8bef-99c6191b966b None None] MFA Rules not processed for user `7bf88e68a7fa4c898e707126aecc8837`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:20:28.244387 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-dffc7aa5-1003-4d68-8bef-99c6191b966b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:20:28.307492 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1442/2896] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:20:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:20:28.319068 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44890082-fc23-41d2-b866-2cb47ca721cf admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:20:28.320776 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-44890082-fc23-41d2-b866-2cb47ca721cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:20:28.342655 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:20:28.379519 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44890082-fc23-41d2-b866-2cb47ca721cf admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:20:28.383173 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f3948f-0d65-466b-8376-9f79f1d952e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44dfe45766d743558ed82e464721cf72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:20:28.383781 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3948f-0d65-466b-8376-9f79f1d952e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:20:28.384063 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3948f-0d65-466b-8376-9f79f1d952e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:20:28.384339 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3948f-0d65-466b-8376-9f79f1d952e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:20:28.393734 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f3948f-0d65-466b-8376-9f79f1d952e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:20:28.395160 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f3948f-0d65-466b-8376-9f79f1d952e8 service cinder] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:20:28.404329 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1455/2897] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:20:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:20:29.765478 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e41bc-c4d8-49a0-8b4e-a00caf9facdd None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:20:29.766055 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e41bc-c4d8-49a0-8b4e-a00caf9facdd None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:20:29.766055 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e41bc-c4d8-49a0-8b4e-a00caf9facdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:20:29.785156 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-135e41bc-c4d8-49a0-8b4e-a00caf9facdd None None] MFA Rules not processed for user `7bf88e68a7fa4c898e707126aecc8837`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:20:29.786352 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-135e41bc-c4d8-49a0-8b4e-a00caf9facdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:20:29.793209 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1443/2898] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:20:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:20:30.267447 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f3948f-0d65-466b-8376-9f79f1d952e8 service cinder] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:20:30.268911 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-45f3948f-0d65-466b-8376-9f79f1d952e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:20:30.351783 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f3948f-0d65-466b-8376-9f79f1d952e8 service cinder] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:20:30.362234 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54773c83-5a9f-4183-8d26-295b16188c89 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61e2c954f9a49abad9f3de17ee39f63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:20:30.362877 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54773c83-5a9f-4183-8d26-295b16188c89 service placement] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:20:30.363151 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54773c83-5a9f-4183-8d26-295b16188c89 service placement] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:20:30.363416 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54773c83-5a9f-4183-8d26-295b16188c89 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:20:30.365287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-54773c83-5a9f-4183-8d26-295b16188c89 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:20:30.377696 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54773c83-5a9f-4183-8d26-295b16188c89 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:20:30.379295 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54773c83-5a9f-4183-8d26-295b16188c89 service placement] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:20:30.391330 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1456/2899] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:20:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:20:39.780138 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:20:40.385962 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=49 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:20:41.947949 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e41bc-c4d8-49a0-8b4e-a00caf9facdd None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:20:41.949379 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-135e41bc-c4d8-49a0-8b4e-a00caf9facdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:20:42.030849 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e41bc-c4d8-49a0-8b4e-a00caf9facdd None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:20:42.036468 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9784f-0c3d-451d-8211-a2b7ecb25e89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e9f085e4f04d118373b8abff27543a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:20:42.037097 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9784f-0c3d-451d-8211-a2b7ecb25e89 service nova] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:20:42.037372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9784f-0c3d-451d-8211-a2b7ecb25e89 service nova] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:20:42.037638 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9784f-0c3d-451d-8211-a2b7ecb25e89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:20:42.039348 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-d1a9784f-0c3d-451d-8211-a2b7ecb25e89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:20:42.051928 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a9784f-0c3d-451d-8211-a2b7ecb25e89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:20:42.053594 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a9784f-0c3d-451d-8211-a2b7ecb25e89 service nova] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:20:42.064874 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1444/2900] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:20:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:20:48.069340 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b8267-c1d0-4ae5-b05f-73305f241b83 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:20:48.069855 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b8267-c1d0-4ae5-b05f-73305f241b83 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:20:48.069855 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b8267-c1d0-4ae5-b05f-73305f241b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:20:48.089201 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-432b8267-c1d0-4ae5-b05f-73305f241b83 None None] MFA Rules not processed for user `3488a7d2258242c0859a105c1d162767`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:20:48.090398 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-432b8267-c1d0-4ae5-b05f-73305f241b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:20:48.145434 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1457/2901] 10.222.0.39 () {58 vars in 1009 bytes} [Fri Aug 5 20:20:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:20:48.192199 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9784f-0c3d-451d-8211-a2b7ecb25e89 service nova] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:20:48.193633 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-d1a9784f-0c3d-451d-8211-a2b7ecb25e89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:20:48.263320 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9784f-0c3d-451d-8211-a2b7ecb25e89 service nova] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:20:48.268844 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b814d996-a89a-4131-9abd-96f1cf424007 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91d3d43cb0bd4ca3b8e8f10e29cc421c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:20:48.269454 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814d996-a89a-4131-9abd-96f1cf424007 service swift] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:20:48.269735 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814d996-a89a-4131-9abd-96f1cf424007 service swift] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:20:48.270011 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814d996-a89a-4131-9abd-96f1cf424007 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:20:48.278443 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b814d996-a89a-4131-9abd-96f1cf424007 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:20:48.280562 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b814d996-a89a-4131-9abd-96f1cf424007 service swift] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:20:48.288976 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1445/2902] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Aug 5 20:20:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 20:20:48.838687 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8014a46-e57b-44d1-b581-f81cc5a47746 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:20:48.838969 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8014a46-e57b-44d1-b581-f81cc5a47746 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:20:48.839241 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8014a46-e57b-44d1-b581-f81cc5a47746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:20:48.858959 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-e8014a46-e57b-44d1-b581-f81cc5a47746 None None] MFA Rules not processed for user `3488a7d2258242c0859a105c1d162767`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:20:48.860161 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-e8014a46-e57b-44d1-b581-f81cc5a47746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:20:48.867334 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1458/2903] 10.222.0.39 () {58 vars in 967 bytes} [Fri Aug 5 20:20:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:20:48.886680 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b814d996-a89a-4131-9abd-96f1cf424007 service swift] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:20:48.898047 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b814d996-a89a-4131-9abd-96f1cf424007 service swift] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:20:48.903372 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61d2b2-4489-4aca-9b2e-d994b0295154 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91d3d43cb0bd4ca3b8e8f10e29cc421c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:20:48.904127 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61d2b2-4489-4aca-9b2e-d994b0295154 service swift] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:20:48.904416 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61d2b2-4489-4aca-9b2e-d994b0295154 service swift] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:20:48.904687 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61d2b2-4489-4aca-9b2e-d994b0295154 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:20:48.913346 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a61d2b2-4489-4aca-9b2e-d994b0295154 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:20:48.914940 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a61d2b2-4489-4aca-9b2e-d994b0295154 service swift] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:20:48.923922 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1446/2904] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Aug 5 20:20:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 20:20:49.151829 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab89c7-3175-4f7f-b104-325905c65f89 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:20:49.152258 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab89c7-3175-4f7f-b104-325905c65f89 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:20:49.152366 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab89c7-3175-4f7f-b104-325905c65f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:20:49.173101 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-63ab89c7-3175-4f7f-b104-325905c65f89 None None] MFA Rules not processed for user `3488a7d2258242c0859a105c1d162767`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:20:49.174321 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-63ab89c7-3175-4f7f-b104-325905c65f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:20:49.181888 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1459/2905] 10.222.0.39 () {58 vars in 1009 bytes} [Fri Aug 5 20:20:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:20:49.194642 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61d2b2-4489-4aca-9b2e-d994b0295154 service swift] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:20:49.205818 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61d2b2-4489-4aca-9b2e-d994b0295154 service swift] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:20:49.211722 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4329e-618b-4c93-a252-b84886c2b19b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91d3d43cb0bd4ca3b8e8f10e29cc421c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:20:49.212349 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4329e-618b-4c93-a252-b84886c2b19b service swift] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:20:49.212695 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4329e-618b-4c93-a252-b84886c2b19b service swift] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:20:49.212996 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4329e-618b-4c93-a252-b84886c2b19b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:20:49.222799 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4329e-618b-4c93-a252-b84886c2b19b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:20:49.224413 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4329e-618b-4c93-a252-b84886c2b19b service swift] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:20:49.236383 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1447/2906] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Aug 5 20:20:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 20:20:49.573534 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f095f2-149d-4899-8bae-18ac2beb6725 None None] REQUEST_METHOD: `POST` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:20:49.575487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f095f2-149d-4899-8bae-18ac2beb6725 None None] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:20:49.575487 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f095f2-149d-4899-8bae-18ac2beb6725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:20:49.593988 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.auth.core [None req-78f095f2-149d-4899-8bae-18ac2beb6725 None None] MFA Rules not processed for user `3488a7d2258242c0859a105c1d162767`. Rule list: `[]` (Enabled: `True`). {{(pid=80478) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:20:49.595192 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-78f095f2-149d-4899-8bae-18ac2beb6725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:20:49.602382 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1460/2907] 10.222.0.39 () {58 vars in 967 bytes} [Fri Aug 5 20:20:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:20:49.614951 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4329e-618b-4c93-a252-b84886c2b19b service swift] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:20:49.626420 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4329e-618b-4c93-a252-b84886c2b19b service swift] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:20:49.631400 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d0dcf-5bad-43bb-b47a-49971a61d019 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91d3d43cb0bd4ca3b8e8f10e29cc421c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:20:49.632022 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d0dcf-5bad-43bb-b47a-49971a61d019 service swift] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:20:49.632302 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d0dcf-5bad-43bb-b47a-49971a61d019 service swift] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:20:49.632598 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d0dcf-5bad-43bb-b47a-49971a61d019 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:20:49.641519 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5d0dcf-5bad-43bb-b47a-49971a61d019 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:20:49.643136 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5d0dcf-5bad-43bb-b47a-49971a61d019 service swift] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:20:49.651380 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1448/2908] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Aug 5 20:20:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 20:20:51.849720 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f095f2-149d-4899-8bae-18ac2beb6725 None None] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:20:51.921782 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f095f2-149d-4899-8bae-18ac2beb6725 None None] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:20:51.927628 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d534032-d2ee-457a-8223-cca4255fd91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:20:51.928479 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d534032-d2ee-457a-8223-cca4255fd91f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:20:51.928877 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d534032-d2ee-457a-8223-cca4255fd91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:20:51.929248 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d534032-d2ee-457a-8223-cca4255fd91f admin admin] PATH_INFO: `/v3/users/7bf88e68a7fa4c898e707126aecc8837` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:20:51.931103 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d534032-d2ee-457a-8223-cca4255fd91f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bf88e68a7fa4c898e707126aecc8837)` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:20:51.934163 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d534032-d2ee-457a-8223-cca4255fd91f admin admin] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:20:51.990155 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-4d534032-d2ee-457a-8223-cca4255fd91f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bf88e68a7fa4c898e707126aecc8837'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:20:51.998569 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-4d534032-d2ee-457a-8223-cca4255fd91f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bf88e68a7fa4c898e707126aecc8837'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:20:52.006974 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-4d534032-d2ee-457a-8223-cca4255fd91f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bf88e68a7fa4c898e707126aecc8837'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:20:52.021793 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-4d534032-d2ee-457a-8223-cca4255fd91f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bf88e68a7fa4c898e707126aecc8837'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:20:52.022532 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.notifications [None req-4d534032-d2ee-457a-8223-cca4255fd91f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bf88e68a7fa4c898e707126aecc8837'} {{(pid=80478) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:20:52.028746 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1461/2909] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Aug 5 20:20:51 2022] DELETE /identity/v3/users/7bf88e68a7fa4c898e707126aecc8837 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:20:52.034081 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00fabb-c870-4205-9da3-c2bbb85e15db None None] REQUEST_METHOD: `POST` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:20:52.034593 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00fabb-c870-4205-9da3-c2bbb85e15db None None] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:20:52.034726 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00fabb-c870-4205-9da3-c2bbb85e15db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:20:52.053685 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.auth.core [None req-fa00fabb-c870-4205-9da3-c2bbb85e15db None None] MFA Rules not processed for user `88c3879e2d344041935d28c7f22fdeeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80477) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:20:52.054901 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-fa00fabb-c870-4205-9da3-c2bbb85e15db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:20:52.085471 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=100 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:20:52.115581 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1449/2910] 10.222.0.39 () {58 vars in 958 bytes} [Fri Aug 5 20:20:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:20:52.126876 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d534032-d2ee-457a-8223-cca4255fd91f admin admin] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:20:52.128338 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-4d534032-d2ee-457a-8223-cca4255fd91f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:20:52.134919 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:20:52.191770 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d534032-d2ee-457a-8223-cca4255fd91f admin admin] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:20:52.195305 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea0505-952a-4026-bd0d-bc05764d2699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1c75368c00e4b0cb7b3163728843479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:20:52.195921 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea0505-952a-4026-bd0d-bc05764d2699 service neutron] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:20:52.196204 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea0505-952a-4026-bd0d-bc05764d2699 service neutron] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:20:52.196479 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea0505-952a-4026-bd0d-bc05764d2699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:20:52.206020 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea0505-952a-4026-bd0d-bc05764d2699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:20:52.207558 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea0505-952a-4026-bd0d-bc05764d2699 service neutron] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:20:52.218992 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1462/2911] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:20:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:20:52.397483 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa00fabb-c870-4205-9da3-c2bbb85e15db None None] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:20:52.399607 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-fa00fabb-c870-4205-9da3-c2bbb85e15db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:20:52.412570 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa00fabb-c870-4205-9da3-c2bbb85e15db None None] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:20:52.417342 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115e878-18e0-4048-afe2-89f85d511ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88c3879e2d344041935d28c7f22fdeeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bb3822e6d1c4f82815dbeda0f6eff52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:20:52.417973 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115e878-18e0-4048-afe2-89f85d511ffb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:20:52.418490 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115e878-18e0-4048-afe2-89f85d511ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:20:52.418795 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115e878-18e0-4048-afe2-89f85d511ffb admin admin] PATH_INFO: `/v3/projects/e6c662e324bb4327af67b163bb03779e` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:20:52.420593 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115e878-18e0-4048-afe2-89f85d511ffb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6c662e324bb4327af67b163bb03779e)` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:20:52.423135 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115e878-18e0-4048-afe2-89f85d511ffb admin admin] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:20:52.471180 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8115e878-18e0-4048-afe2-89f85d511ffb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6c662e324bb4327af67b163bb03779e', 'request_id': 'req-8115e878-18e0-4048-afe2-89f85d511ffb'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:20:52.478403 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8115e878-18e0-4048-afe2-89f85d511ffb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6c662e324bb4327af67b163bb03779e', 'request_id': 'req-8115e878-18e0-4048-afe2-89f85d511ffb'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:20:52.484987 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8115e878-18e0-4048-afe2-89f85d511ffb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6c662e324bb4327af67b163bb03779e', 'request_id': 'req-8115e878-18e0-4048-afe2-89f85d511ffb'} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:20:52.495815 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8115e878-18e0-4048-afe2-89f85d511ffb admin admin] The token cache is being invalidate because project e6c662e324bb4327af67b163bb03779e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80477) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:20:52.496100 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8115e878-18e0-4048-afe2-89f85d511ffb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:20:52.496767 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.notifications [None req-8115e878-18e0-4048-afe2-89f85d511ffb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80477) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:20:52.498355 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1450/2912] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Aug 5 20:20:52 2022] DELETE /identity/v3/projects/e6c662e324bb4327af67b163bb03779e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:20:57.991426 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea0505-952a-4026-bd0d-bc05764d2699 service neutron] Authenticating user token {{(pid=80478) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:20:57.992898 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-63ea0505-952a-4026-bd0d-bc05764d2699 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:20:58.136690 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea0505-952a-4026-bd0d-bc05764d2699 service neutron] Validating token access rules against request {{(pid=80478) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:20:58.182080 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=100 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:20:58.202287 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69133a-69fc-4345-82d0-ba0be2bc986f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61e2c954f9a49abad9f3de17ee39f63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80478) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:20:58.202786 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69133a-69fc-4345-82d0-ba0be2bc986f service placement] REQUEST_METHOD: `GET` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:20:58.203043 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69133a-69fc-4345-82d0-ba0be2bc986f service placement] SCRIPT_NAME: `/identity` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:20:58.203723 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69133a-69fc-4345-82d0-ba0be2bc986f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80478) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:20:58.205247 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.fernet_utils [None req-9d69133a-69fc-4345-82d0-ba0be2bc986f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80478) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:20:58.217927 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69133a-69fc-4345-82d0-ba0be2bc986f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:20:58.219642 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69133a-69fc-4345-82d0-ba0be2bc986f service placement] RBAC: Authorization granted {{(pid=80478) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:20:58.230203 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: [pid: 80478|app: 0|req: 1463/2913] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:20:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:21:02.492899 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=38,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:21:08.225826 dv-cinder-852210-2-iscsi devstack@keystone.service[80478]: DEBUG dbcounter [-] [80478] Writing DB stats keystone:SELECT=13 {{(pid=80478) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:26:03.969774 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115e878-18e0-4048-afe2-89f85d511ffb admin admin] Authenticating user token {{(pid=80477) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:26:03.971153 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.fernet_utils [None req-8115e878-18e0-4048-afe2-89f85d511ffb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80477) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:26:04.010444 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115e878-18e0-4048-afe2-89f85d511ffb admin admin] Validating token access rules against request {{(pid=80477) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:26:04.015449 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5008093-d542-43ea-9d55-60b3a4346cf0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61e2c954f9a49abad9f3de17ee39f63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80379e2104644e98a2e85d4e4af1b365', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80477) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:26:04.016049 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5008093-d542-43ea-9d55-60b3a4346cf0 service placement] REQUEST_METHOD: `GET` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:26:04.016323 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5008093-d542-43ea-9d55-60b3a4346cf0 service placement] SCRIPT_NAME: `/identity` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:26:04.016639 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5008093-d542-43ea-9d55-60b3a4346cf0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80477) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:26:04.025487 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5008093-d542-43ea-9d55-60b3a4346cf0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:26:04.027040 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5008093-d542-43ea-9d55-60b3a4346cf0 service placement] RBAC: Authorization granted {{(pid=80477) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:26:04.036846 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: [pid: 80477|app: 0|req: 1451/2914] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Aug 5 20:26:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:26:14.032658 dv-cinder-852210-2-iscsi devstack@keystone.service[80477]: DEBUG dbcounter [-] [80477] Writing DB stats keystone:SELECT=17 {{(pid=80477) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}